|
2021-06-14
§
|
| 06:32 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1180 (re)pooling @ 25%: Repool db1180 after schema change', diff saved to https://phabricator.wikimedia.org/P16448 and previous config saved to /var/cache/conftool/dbconfig/20210614-063231-root.json |
[production] |
| 06:25 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1180 for schema change', diff saved to https://phabricator.wikimedia.org/P16447 and previous config saved to /var/cache/conftool/dbconfig/20210614-062554-marostegui.json |
[production] |
| 06:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3316 (re)pooling @ 100%: Repool db1113:3316 after schema change', diff saved to https://phabricator.wikimedia.org/P16446 and previous config saved to /var/cache/conftool/dbconfig/20210614-061226-root.json |
[production] |
| 06:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 100%: Repool db1099:3311 after schema change', diff saved to https://phabricator.wikimedia.org/P16445 and previous config saved to /var/cache/conftool/dbconfig/20210614-060119-root.json |
[production] |
| 05:57 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3316 (re)pooling @ 75%: Repool db1113:3316 after schema change', diff saved to https://phabricator.wikimedia.org/P16444 and previous config saved to /var/cache/conftool/dbconfig/20210614-055723-root.json |
[production] |
| 05:46 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 75%: Repool db1099:3311 after schema change', diff saved to https://phabricator.wikimedia.org/P16443 and previous config saved to /var/cache/conftool/dbconfig/20210614-054615-root.json |
[production] |
| 05:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3316 (re)pooling @ 50%: Repool db1113:3316 after schema change', diff saved to https://phabricator.wikimedia.org/P16442 and previous config saved to /var/cache/conftool/dbconfig/20210614-054219-root.json |
[production] |
| 05:31 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 50%: Repool db1099:3311 after schema change', diff saved to https://phabricator.wikimedia.org/P16441 and previous config saved to /var/cache/conftool/dbconfig/20210614-053112-root.json |
[production] |
| 05:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3316 (re)pooling @ 25%: Repool db1113:3316 after schema change', diff saved to https://phabricator.wikimedia.org/P16440 and previous config saved to /var/cache/conftool/dbconfig/20210614-052715-root.json |
[production] |
| 05:19 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1113:3316 for schema change', diff saved to https://phabricator.wikimedia.org/P16439 and previous config saved to /var/cache/conftool/dbconfig/20210614-051930-marostegui.json |
[production] |
| 05:16 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 25%: Repool db1099:3311 after schema change', diff saved to https://phabricator.wikimedia.org/P16438 and previous config saved to /var/cache/conftool/dbconfig/20210614-051608-root.json |
[production] |
| 05:15 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1099:3311 for schema change', diff saved to https://phabricator.wikimedia.org/P16437 and previous config saved to /var/cache/conftool/dbconfig/20210614-051522-marostegui.json |
[production] |
|
2021-06-13
§
|
| 19:10 |
<Krinkle> |
Migrate CVNBot26,27,28,29 to Libera |
[cvn] |
| 19:04 |
<Krinkle> |
Migrate CVNBot21,22,23,24,25 to Libera |
[cvn] |
| 18:54 |
<Krinkle> |
flags #cvn-bots Operator873 local_op (same as Freenode, confirmed account) |
[cvn] |
| 18:52 |
<Krinkle> |
Migrate CVNBot16,17,18,19,20 to Libera |
[cvn] |
| 18:47 |
<Krinkle> |
Migrate Cubbie (now CVNBot11), and CVNBot13,14,15 to Libera |
[cvn] |
| 18:44 |
<Reedy> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/699567 |
[releng] |
| 18:41 |
<Krinkle> |
Migrate CVNBot6,7,8,9,10 to Libera |
[cvn] |
| 18:33 |
<Krinkle> |
Migrate CVNBot4,5 to Libera |
[cvn] |
| 18:29 |
<Krinkle> |
Migrate CVNBot1,2,3 to Libera |
[cvn] |
| 18:21 |
<Krinkle> |
Migrate CVNBot12 to Libera |
[cvn] |
| 18:12 |
<Reedy> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/699541 |
[releng] |
| 18:05 |
<Krinkle> |
flags #cvn-wp-nl natuur12 local_op (same as Freenode, confirmed account) |
[cvn] |
| 18:02 |
<Krinkle> |
Contacted operator of mastiBot (#cvn-wp-pl) to move their bot to Libera. |
[cvn] |
| 16:55 |
<Krinkle> |
flags #cvn-wikidata [1997kB] voiced (same as on Freenode, confirmed account) |
[cvn] |
| 12:54 |
<wm-bot> |
<peterbowman> Deploy 78226e6: add MissingPlwikinewsRefsOnPlwiki servlet |
[tools.pbbot] |
| 08:15 |
<majavah> |
clear grid error state from tools-sgeexec-0907, tools-sgeexec-0916, tools-sgeexec-0940 |
[tools] |
| 03:08 |
<Krinkle> |
Applied cvn/bot/* cloak for CVNBots-A, CVNBots-B to CVNBots-F |
[cvn] |
| 03:08 |
<Krinkle> |
Applied cvn/bot/* cloak for cvn-clerkbot |
[cvn] |
| 03:08 |
<Krinkle> |
Migrated cvn-clerkbot to Libera (same config and channel list) |
[cvn] |
| 00:42 |
<Krinkle> |
flags #cvn-zho WhitePhosphorus local_op |
[cvn] |
| 00:39 |
<Krinkle> |
flags #cvn-zh-sw WhitePhosphorus local_op |
[cvn] |
|
2021-06-12
§
|
| 23:59 |
<Krinkle> |
wm-bot has been migrated to Libera (logging #countervandalism, and serving #cvn-es-scan, #cvn-wp-fa, #cvn-zh-sw and #cvn-zho) |
[cvn] |
| 23:56 |
<MacFan4000> |
copied channel config for #cvn-es-scan, #cvn-wp-fa and #countervandalism |
[wm-bot] |
| 19:27 |
<valhallasw> |
Deployed 081c4e1df0cf8c7e3525f12f194197c2fe1b53a7 |
[tools.forrestbot] |
| 14:42 |
<majavah> |
sync hiera key prometheus_nodes to match tools |
[toolsbeta] |
| 14:39 |
<majavah> |
remove nonexistent tools-prometheus-04 and add tools-prometheus-05 to hiera key "prometheus_nodes" |
[tools] |
| 13:53 |
<majavah> |
create empty bullseye-{tools,toolsbeta} repositories on tools-services-05 aptly |
[tools] |
| 13:49 |
<rzl@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on 6 hosts with reason: alert noise, no impact, x2 is unused |
[production] |
| 13:49 |
<rzl@cumin1001> |
START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on 6 hosts with reason: alert noise, no impact, x2 is unused |
[production] |
|
2021-06-11
§
|
| 23:37 |
<mutante> |
removing firewall hole for mgmt networks to install* because it turned out it cant be used for firmware upgrades |
[production] |
| 22:08 |
<brennen> |
gitlab.wikimedia.org currently up with recommended config applied; test data deleted; users can register but not create projects. brennen, dancy, and thcipriani currently marked as admins. may need to reset data again, but hopefully not. |
[production] |
| 21:27 |
<pt1979@cumin2002> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on pc2014.codfw.wmnet with reason: REIMAGE |
[production] |
| 21:25 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on pc2014.codfw.wmnet with reason: REIMAGE |
[production] |
| 21:01 |
<pt1979@cumin2002> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on pc2013.codfw.wmnet with reason: REIMAGE |
[production] |
| 20:59 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on pc2013.codfw.wmnet with reason: REIMAGE |
[production] |
| 20:49 |
<brennen> |
gitlab1001: resetting application data, re-running ansible playbook |
[releng] |
| 20:04 |
<pt1979@cumin2002> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on pc2012.codfw.wmnet with reason: REIMAGE |
[production] |
| 20:02 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on pc2012.codfw.wmnet with reason: REIMAGE |
[production] |