|
2024-08-19
§
|
| 08:34 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db2136', diff saved to https://phabricator.wikimedia.org/P67371 and previous config saved to /var/cache/conftool/dbconfig/20240819-083439-root.json |
[production] |
| 08:33 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox |
[production] |
| 08:32 |
<ayounsi@cumin1002> |
START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox |
[production] |
| 08:32 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox-canary |
[production] |
| 08:31 |
<ayounsi@cumin1002> |
START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox-canary |
[production] |
| 08:18 |
<brouberol@cumin1002> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 08:18 |
<brouberol@cumin1002> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Adding AAAA field to snapshot1010 and dumpsdata1003 - brouberol@cumin1002" |
[production] |
| 08:18 |
<brouberol@cumin1002> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Adding AAAA field to snapshot1010 and dumpsdata1003 - brouberol@cumin1002" |
[production] |
| 08:14 |
<brouberol@cumin1002> |
START - Cookbook sre.dns.netbox |
[production] |
| 07:25 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox |
[production] |
| 07:25 |
<ayounsi@cumin1002> |
START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox |
[production] |
| 07:25 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox-canary |
[production] |
| 07:24 |
<ayounsi@cumin1002> |
START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox-canary |
[production] |
| 07:16 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox |
[production] |
| 07:16 |
<ayounsi@cumin1002> |
START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox |
[production] |
| 07:14 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox-canary |
[production] |
| 07:14 |
<ayounsi@cumin1002> |
START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox-canary |
[production] |
| 07:00 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1195 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P67370 and previous config saved to /var/cache/conftool/dbconfig/20240819-070034-root.json |
[production] |
| 06:45 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1195 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P67369 and previous config saved to /var/cache/conftool/dbconfig/20240819-064528-root.json |
[production] |
| 06:30 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1195 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P67368 and previous config saved to /var/cache/conftool/dbconfig/20240819-063023-root.json |
[production] |
| 06:15 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1195 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P67367 and previous config saved to /var/cache/conftool/dbconfig/20240819-061517-root.json |
[production] |
| 06:00 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1195 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P67366 and previous config saved to /var/cache/conftool/dbconfig/20240819-060011-root.json |
[production] |
| 05:45 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1195 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P67365 and previous config saved to /var/cache/conftool/dbconfig/20240819-054506-root.json |
[production] |
| 05:30 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1195 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P67364 and previous config saved to /var/cache/conftool/dbconfig/20240819-053000-root.json |
[production] |
| 05:25 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1195.eqiad.wmnet with reason: Upgrade to 10.6.19 |
[production] |
| 05:24 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 1:00:00 on db1195.eqiad.wmnet with reason: Upgrade to 10.6.19 |
[production] |
| 05:23 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db1195 T372536', diff saved to https://phabricator.wikimedia.org/P67363 and previous config saved to /var/cache/conftool/dbconfig/20240819-052352-root.json |
[production] |
|
2024-08-18
§
|
| 22:03 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Setting back s4 to RW', diff saved to https://phabricator.wikimedia.org/P67362 and previous config saved to /var/cache/conftool/dbconfig/20240818-220355-ladsgroup.json |
[production] |
| 22:00 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Set s4 as read-only ', diff saved to https://phabricator.wikimedia.org/P67361 and previous config saved to /var/cache/conftool/dbconfig/20240818-220043-ladsgroup.json |
[production] |
| 20:54 |
<kamila@cumin2002> |
dbctl commit (dc=all): 'Setting s4 back to RW', diff saved to https://phabricator.wikimedia.org/P67360 and previous config saved to /var/cache/conftool/dbconfig/20240818-205410-kamila.json |
[production] |
| 20:50 |
<kamila@cumin2002> |
dbctl commit (dc=all): 'Set s4 as read-only due to overload', diff saved to https://phabricator.wikimedia.org/P67359 and previous config saved to /var/cache/conftool/dbconfig/20240818-205024-kamila.json |
[production] |
| 15:56 |
<wmbot~lucaswerkmeister@tools-bastion-13> |
kubectl rollout restart deployment flr # after reports of FlickreviewR 2 not working on IRC |
[tools.yifeibot] |
| 14:19 |
<wmbot~bsadowski1@tools-bastion-13> |
Restarted StewardBot/StewardBot because of a connection loss |
[tools.stewardbots] |
| 12:03 |
<wmbot~lucaswerkmeister@tools-bastion-13> |
deployed 309b33b80b (l10n updates: pl, tg) |
[tools.lexeme-forms] |
| 12:02 |
<wmbot~lucaswerkmeister@tools-bastion-13> |
deployed 6deace1e36 (Italian masculine+feminine nouns, dependency upgrades) |
[tools.lexeme-forms] |
|
2024-08-17
§
|
| 11:33 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db2152 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P67358 and previous config saved to /var/cache/conftool/dbconfig/20240817-113358-root.json |
[production] |
| 11:18 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db2152 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P67357 and previous config saved to /var/cache/conftool/dbconfig/20240817-111852-root.json |
[production] |
| 11:03 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db2152 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P67356 and previous config saved to /var/cache/conftool/dbconfig/20240817-110347-root.json |
[production] |
| 10:48 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db2152 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P67355 and previous config saved to /var/cache/conftool/dbconfig/20240817-104841-root.json |
[production] |
| 10:33 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db2152 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P67354 and previous config saved to /var/cache/conftool/dbconfig/20240817-103336-root.json |
[production] |
| 10:18 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db2152 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P67353 and previous config saved to /var/cache/conftool/dbconfig/20240817-101831-root.json |
[production] |
| 10:03 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db2152 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P67352 and previous config saved to /var/cache/conftool/dbconfig/20240817-100325-root.json |
[production] |
| 09:53 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db2154 (T367856)', diff saved to https://phabricator.wikimedia.org/P67351 and previous config saved to /var/cache/conftool/dbconfig/20240817-095320-marostegui.json |
[production] |
| 09:53 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 7:00:00 on db2154.codfw.wmnet with reason: Maintenance |
[production] |
| 09:53 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 2 days, 7:00:00 on db2154.codfw.wmnet with reason: Maintenance |
[production] |
| 09:52 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 7:00:00 on db2152.codfw.wmnet with reason: Maintenance |
[production] |
| 09:52 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 2 days, 7:00:00 on db2152.codfw.wmnet with reason: Maintenance |
[production] |
| 03:23 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) |
[admin] |
| 03:22 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.ceph.osd.undrain_node |
[admin] |