5301-5350 of 10000 results (119ms)
2024-10-01 §
10:06 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host an-conf1004.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:04 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host irc1003.wikimedia.org [production]
10:02 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-conf1004.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:01 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host an-conf1004.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:00 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host irc1003.wikimedia.org [production]
09:59 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-conf1004.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
09:57 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host an-conf1004.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
09:24 <jmm@deploy2002> Finished scap sync-world: Backport for [[gerrit:1050261|Remove irc1001/irc2001 from mediawiki-config and add irc1003 (T331702 T376014)]] (duration: 08m 07s) [production]
09:19 <jmm@deploy2002> jmm: Continuing with sync [production]
09:19 <fnegri@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 14 days, 0:00:00 on cloudvirt1063.eqiad.wmnet with reason: cloudvirt1063 needs maintenance T375223 [production]
09:18 <fnegri@cumin1002> START - Cookbook sre.hosts.downtime for 14 days, 0:00:00 on cloudvirt1063.eqiad.wmnet with reason: cloudvirt1063 needs maintenance T375223 [production]
09:18 <jmm@deploy2002> jmm: Backport for [[gerrit:1050261|Remove irc1001/irc2001 from mediawiki-config and add irc1003 (T331702 T376014)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
09:15 <jmm@deploy2002> Started scap sync-world: Backport for [[gerrit:1050261|Remove irc1001/irc2001 from mediawiki-config and add irc1003 (T331702 T376014)]] [production]
09:07 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1177 (T367856)', diff saved to https://phabricator.wikimedia.org/P69437 and previous config saved to /var/cache/conftool/dbconfig/20241001-090708-ladsgroup.json [production]
09:07 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 12:00:00 on db1177.eqiad.wmnet with reason: Maintenance [production]
09:06 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 2 days, 12:00:00 on db1177.eqiad.wmnet with reason: Maintenance [production]
09:06 <ladsgroup@cumin1002> END (ERROR) - Cookbook sre.hosts.downtime (exit_code=97) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance [production]
09:06 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance [production]
09:06 <ladsgroup@cumin1002> END (ERROR) - Cookbook sre.hosts.downtime (exit_code=97) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance [production]
09:06 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance [production]
08:58 <hashar@deploy2002> rebuilt and synchronized wikiversions files: group0 to 1.43.0-wmf.25 refs T375656 [production]
08:46 <urbanecm@deploy2002> Finished scap sync-world: Backport for [[gerrit:1076969|DatabaseMentorStore: Cast user IDs to integers before looking them up (T375784)]] (duration: 06m 58s) [production]
08:39 <urbanecm@deploy2002> Started scap sync-world: Backport for [[gerrit:1076969|DatabaseMentorStore: Cast user IDs to integers before looking them up (T375784)]] [production]
07:58 <jynus@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on pc1013.eqiad.wmnet with reason: T375382 [production]
07:54 <jynus@cumin1002> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on pc1013.eqiad.wmnet with reason: T375382 [production]
07:43 <jynus@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on db1246.eqiad.wmnet with reason: T374215 [production]
07:39 <jynus@cumin1002> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on db1246.eqiad.wmnet with reason: T374215 [production]
07:34 <kartik@deploy2002> Finished scap sync-world: Backport for [[gerrit:1076661|Add namespace aliases for scn.wikipedia (T375979)]] (duration: 10m 05s) [production]
07:30 <kartik@deploy2002> kartik, melos: Continuing with sync [production]
07:26 <kartik@deploy2002> kartik, melos: Backport for [[gerrit:1076661|Add namespace aliases for scn.wikipedia (T375979)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
07:24 <kartik@deploy2002> Started scap sync-world: Backport for [[gerrit:1076661|Add namespace aliases for scn.wikipedia (T375979)]] [production]
07:21 <kartik@deploy2002> Finished scap sync-world: Backport for [[gerrit:1075189|Enable translation settings banner for Test wikipedia (T372460)]] (duration: 18m 15s) [production]
07:14 <kartik@deploy2002> kartik, abi: Continuing with sync [production]
07:09 <kartik@deploy2002> kartik, abi: Backport for [[gerrit:1075189|Enable translation settings banner for Test wikipedia (T372460)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
07:03 <kartik@deploy2002> Started scap sync-world: Backport for [[gerrit:1075189|Enable translation settings banner for Test wikipedia (T372460)]] [production]
06:47 <root@cumin2002> END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Luke Bowmaker out of all services on: 705 hosts [production]
06:47 <root@cumin2002> START - Cookbook sre.idm.logout Logging Luke Bowmaker out of all services on: 705 hosts [production]
06:47 <root@cumin2002> END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Luke Bowmaker out of all services on: 1497 hosts [production]
06:46 <root@cumin2002> START - Cookbook sre.idm.logout Logging Luke Bowmaker out of all services on: 1497 hosts [production]
06:44 <XioNoX> cr3-ulsfo> request vmhost snapshot - T375345 [production]
04:01 <mwpresync@deploy2002> Pruned MediaWiki: 1.43.0-wmf.22 (duration: 00m 58s) [production]
03:51 <mwpresync@deploy2002> Finished scap sync-world: testwikis to 1.43.0-wmf.25 refs T375656 (duration: 48m 36s) [production]
03:02 <mwpresync@deploy2002> Started scap sync-world: testwikis to 1.43.0-wmf.25 refs T375656 [production]
02:47 <eileen> civicrm upgraded from cf27c789 to 28fd5e3b [production]
02:17 <ejegg> email preference center upgraded from 8ff002ef to e88750e6 [production]
02:16 <ejegg> payments-wiki upgraded from 8d3b8e94 to e88750e6 [production]
2024-09-30 §
22:44 <jhathaway@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host sretest1002.eqiad.wmnet with OS bookworm [production]
22:00 <zabe@deploy2002> Finished scap sync-world: Backport for [[gerrit:1076760|s3: Reduce revision-slots cache expiry to 60 seconds (T183490)]] (duration: 06m 54s) [production]
21:55 <zabe@deploy2002> zabe: Continuing with sync [production]
21:55 <zabe@deploy2002> zabe: Backport for [[gerrit:1076760|s3: Reduce revision-slots cache expiry to 60 seconds (T183490)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]