9551-9600 of 10000 results (158ms)
2024-09-20 §
13:20 <moritzm> uploaded debmonitor-client 0.4.0-3 for buster/bullseye/bookworm to apt.wikimedia.org T216832 [production]
13:09 <jiji@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on mw2425.codfw.wmnet with reason: reimage [production]
13:09 <jiji@cumin1002> START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on mw2425.codfw.wmnet with reason: reimage [production]
12:45 <jiji@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host mw2424.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTARTand with Dell SCP reboot policy GRACEFUL [production]
12:38 <jiji@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2425.codfw.wmnet with reason: reimage [production]
12:37 <jiji@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw2425.codfw.wmnet with reason: reimage [production]
12:36 <jiji@cumin1002> START - Cookbook sre.hosts.provision for host mw2424.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTARTand with Dell SCP reboot policy GRACEFUL [production]
12:24 <jiji@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host mw2424.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTARTand with Dell SCP reboot policy GRACEFUL [production]
12:18 <jiji@cumin1002> START - Cookbook sre.hosts.provision for host mw2424.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTARTand with Dell SCP reboot policy GRACEFUL [production]
12:16 <jiji@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host mw2424.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTARTand with Dell SCP reboot policy GRACEFUL [production]
12:11 <jiji@cumin1002> START - Cookbook sre.hosts.provision for host mw2424.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTARTand with Dell SCP reboot policy GRACEFUL [production]
12:07 <jiji@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2425.codfw.wmnet with reason: reimage [production]
12:07 <jiji@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw2425.codfw.wmnet with reason: reimage [production]
11:58 <jiji@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host mw2424.codfw.wmnet [production]
11:57 <jiji@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host mw2424.codfw.wmnet [production]
11:46 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on wikikube-worker2092.codfw.wmnet with reason: Degraded RAID [production]
11:46 <cgoubert@cumin1002> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on wikikube-worker2092.codfw.wmnet with reason: Degraded RAID [production]
11:18 <moritzm> installing links2 updates from Bullseye point release [production]
11:11 <moritzm> installing nano updates from Bullseye point update [production]
10:11 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device lsw1-f3-eqiad [production]
10:11 <ayounsi@cumin1002> START - Cookbook sre.network.tls for network device lsw1-f3-eqiad [production]
10:08 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device lsw1-f2-eqiad [production]
10:08 <ayounsi@cumin1002> START - Cookbook sre.network.tls for network device lsw1-f2-eqiad [production]
10:08 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device lsw1-e3-eqiad [production]
10:07 <ayounsi@cumin1002> START - Cookbook sre.network.tls for network device lsw1-e3-eqiad [production]
10:07 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device lsw1-e2-eqiad [production]
10:07 <ayounsi@cumin1002> START - Cookbook sre.network.tls for network device lsw1-e2-eqiad [production]
10:03 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device lsw1-e1-eqiad [production]
10:03 <ayounsi@cumin1002> START - Cookbook sre.network.tls for network device lsw1-e1-eqiad [production]
09:42 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device cloudsw1-f4-eqiad [production]
09:40 <ayounsi@cumin1002> START - Cookbook sre.network.tls for network device cloudsw1-f4-eqiad [production]
09:38 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device cloudsw1-e4-eqiad [production]
09:35 <ayounsi@cumin1002> START - Cookbook sre.network.tls for network device cloudsw1-e4-eqiad [production]
09:34 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device cloudsw1-b1-codfw [production]
09:32 <ayounsi@cumin1002> START - Cookbook sre.network.tls for network device cloudsw1-b1-codfw [production]
08:54 <btullis@cumin1002> END (PASS) - Cookbook sre.hadoop.reboot-workers (exit_code=0) for Hadoop analytics cluster [production]
08:37 <arnaudb@cumin1002> dbctl commit (dc=all): 'depool es1022 - T375257', diff saved to https://phabricator.wikimedia.org/P69377 and previous config saved to /var/cache/conftool/dbconfig/20240920-083722-arnaudb.json [production]
08:32 <jgleeson|away> payments config changed from eee33e37 to e140fab5 [production]
07:50 <tappof> T375085 testing mtail 3.0.9 using debian testing package on centrallog2002 [production]
07:33 <moritzm> rebalance ganeti group D following the various switch maintenances T370630 [production]
07:28 <jelto@cumin1002> END (FAIL) - Cookbook sre.gitlab.upgrade (exit_code=99) on GitLab host gitlab1003.wikimedia.org with reason: test I946dd0b73b6be2d6b8093f03550f78d76188b92b with dummy upgrade [production]
07:27 <jelto@cumin1002> START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1003.wikimedia.org with reason: test I946dd0b73b6be2d6b8093f03550f78d76188b92b with dummy upgrade [production]
07:18 <vgutierrez> rolling upgrade of purged on magru, drmrs, esams and eqiad - T334078 [production]
2024-09-19 §
21:15 <Dreamy_Jazz> Evening UTC backport window done [production]
21:15 <dreamyjazz@deploy1003> Finished scap sync-world: Backport for [[gerrit:1073871|Re-order arguments to DataAccess::addTrackingCategory]], [[gerrit:1074253|Add a "duplicate-ids" lint category (T200517)]] (duration: 29m 20s) [production]
21:02 <dreamyjazz@deploy1003> dreamyjazz, cscott: Continuing with sync [production]
21:00 <dreamyjazz@deploy1003> dreamyjazz, cscott: Backport for [[gerrit:1073871|Re-order arguments to DataAccess::addTrackingCategory]], [[gerrit:1074253|Add a "duplicate-ids" lint category (T200517)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:45 <dreamyjazz@deploy1003> Started scap sync-world: Backport for [[gerrit:1073871|Re-order arguments to DataAccess::addTrackingCategory]], [[gerrit:1074253|Add a "duplicate-ids" lint category (T200517)]] [production]
20:36 <Dreamy_Jazz> Running `foreachwikiindblist group1.dblist extensions/CheckUser/maintenance/populateCentralCheckUserIndexTables.php` on a tmux session for T375203 [production]
20:34 <dreamyjazz@deploy1003> Finished scap sync-world: Backport for [[gerrit:1074241|Add ::caller to queries in populateCentralCheckUserIndexTables.php (T375221)]] (duration: 06m 50s) [production]