3451-3500 of 10000 results (118ms)
2024-01-15 §
11:09 <jiji@cumin1002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc1037.eqiad.wmnet [production]
11:08 <btullis@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on 8 hosts with reason: Bringing new nameservers into service [production]
11:08 <btullis@cumin1002> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on 8 hosts with reason: Bringing new nameservers into service [production]
11:08 <btullis@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on 97 hosts with reason: Bringing new nameservers into service [production]
11:07 <btullis@cumin1002> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on 97 hosts with reason: Bringing new nameservers into service [production]
11:03 <jiji@cumin1002> START - Cookbook sre.hosts.reboot-single for host mc1037.eqiad.wmnet [production]
10:58 <jiji@cumin1002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc-gp1002.eqiad.wmnet [production]
10:51 <jiji@cumin1002> START - Cookbook sre.hosts.reboot-single for host mc-gp1002.eqiad.wmnet [production]
10:48 <moritzm> installing systemd bugfix updates from Bullseye point release [production]
10:30 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host mc1037.eqiad.wmnet [production]
10:13 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-host for host mc1037.eqiad.wmnet [production]
10:08 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host mc-gp1002.eqiad.wmnet [production]
10:02 <ladsgroup@deploy2002> Finished scap: Backport for [[gerrit:990424|SecurePoll: Adding updated voterlist files (T349263)]] (duration: 16m 04s) [production]
09:58 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-host for host mc-gp1002.eqiad.wmnet [production]
09:56 <ladsgroup@deploy2002> ladsgroup: Continuing with sync [production]
09:48 <ladsgroup@deploy2002> ladsgroup: Backport for [[gerrit:990424|SecurePoll: Adding updated voterlist files (T349263)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
09:46 <ladsgroup@deploy2002> Started scap: Backport for [[gerrit:990424|SecurePoll: Adding updated voterlist files (T349263)]] [production]
09:16 <pfischer@deploy2002> helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
09:16 <pfischer@deploy2002> helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply [production]
09:15 <pfischer@deploy2002> helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
09:15 <pfischer@deploy2002> helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply [production]
09:15 <pfischer@deploy2002> helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
09:14 <pfischer@deploy2002> helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply [production]
08:45 <filippo@deploy2002> Finished deploy [performance/arc-lamp@67389a0]: (no justification provided) (duration: 00m 05s) [production]
08:45 <filippo@deploy2002> Started deploy [performance/arc-lamp@67389a0]: (no justification provided) [production]
08:23 <dcausse@deploy2002> Finished scap: Backport for [[gerrit:990029|enable page_rerender for 5th batch of wikis (T351503)]] (duration: 11m 40s) [production]
08:17 <dcausse@deploy2002> pfischer and dcausse: Continuing with sync [production]
08:13 <dcausse@deploy2002> pfischer and dcausse: Backport for [[gerrit:990029|enable page_rerender for 5th batch of wikis (T351503)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
08:12 <dcausse@deploy2002> Started scap: Backport for [[gerrit:990029|enable page_rerender for 5th batch of wikis (T351503)]] [production]
04:57 <andrewbogott> restarting wikitech-static, oom [production]
2024-01-14 §
15:47 <taavi@deploy2002> Finished scap: Backport for [[gerrit:990396|Log IpReputation channel as debug (T354928)]] (duration: 26m 49s) [production]
15:36 <taavi@deploy2002> taavi: Continuing with sync [production]
15:35 <taavi@deploy2002> taavi: Backport for [[gerrit:990396|Log IpReputation channel as debug (T354928)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
15:20 <taavi@deploy2002> Started scap: Backport for [[gerrit:990396|Log IpReputation channel as debug (T354928)]] [production]
15:01 <andrewbogott> manually emptying /srv/mediawiki/images/wikitech/archive on wikitech-static; the maintenance script didn't do it and the host is failing due to a full disk [production]
15:01 <andrewbogott> running deleteArchivedFiles.php on wikitech-static [production]
2024-01-12 §
23:49 <dzahn@cumin1001> END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Conniecc1 out of all services on: 2213 hosts [production]
23:47 <dzahn@cumin1001> START - Cookbook sre.idm.logout Logging Conniecc1 out of all services on: 2213 hosts [production]
22:52 <dzahn@cumin1001> END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Conniecc1 out of all services on: 2213 hosts [production]
22:51 <dzahn@cumin1001> START - Cookbook sre.idm.logout Logging Conniecc1 out of all services on: 2213 hosts [production]
22:29 <dzahn@cumin1001> END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Conniecc1 out of all services on: 2213 hosts [production]
22:28 <dzahn@cumin1001> START - Cookbook sre.idm.logout Logging Conniecc1 out of all services on: 2213 hosts [production]
18:07 <mutante> aphlict1002 - systemctl start logrotate [production]
17:18 <tchanders@deploy2002> helmfile [codfw] DONE helmfile.d/services/ipoid: apply [production]
17:18 <tchanders@deploy2002> helmfile [codfw] START helmfile.d/services/ipoid: apply [production]
17:17 <tchanders@deploy2002> helmfile [eqiad] DONE helmfile.d/services/ipoid: apply [production]
17:16 <tchanders@deploy2002> helmfile [eqiad] START helmfile.d/services/ipoid: apply [production]
17:10 <tchanders@deploy2002> helmfile [staging] DONE helmfile.d/services/ipoid: apply [production]
17:09 <tchanders@deploy2002> helmfile [staging] START helmfile.d/services/ipoid: apply [production]
16:52 <cgoubert@deploy2002> helmfile [codfw] [main] DONE helmfile.d/services/mw-jobrunner : sync [production]