4201-4250 of 10000 results (83ms)
2022-11-16 ยง
10:30 <urbanecm> Run `mwscript extensions/GrowthExperiments/maintenance/updateIsActiveFlagForMentees.php` for all wikis in growthexperiments.dblist at mwmaint1002 (T318457) [production]
10:29 <jynus> restarting apache on lists.wm.o [production]
10:16 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2170:3311 (T318605)', diff saved to https://phabricator.wikimedia.org/P39917 and previous config saved to /var/cache/conftool/dbconfig/20221116-101653-ladsgroup.json [production]
10:16 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2170.codfw.wmnet with reason: Maintenance [production]
10:16 <kevinbazira@deploy1002> Finished deploy [ores/deploy@0114799]: T319373 (duration: 10m 51s) [production]
10:16 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2170.codfw.wmnet with reason: Maintenance [production]
10:16 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2167:3311 (T318605)', diff saved to https://phabricator.wikimedia.org/P39916 and previous config saved to /var/cache/conftool/dbconfig/20221116-101631-ladsgroup.json [production]
10:05 <kevinbazira@deploy1002> Started deploy [ores/deploy@0114799]: T319373 [production]
10:01 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2167:3311', diff saved to https://phabricator.wikimedia.org/P39915 and previous config saved to /var/cache/conftool/dbconfig/20221116-100125-ladsgroup.json [production]
10:00 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1134 (T318605)', diff saved to https://phabricator.wikimedia.org/P39914 and previous config saved to /var/cache/conftool/dbconfig/20221116-100027-ladsgroup.json [production]
10:00 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1134.eqiad.wmnet with reason: Maintenance [production]
09:59 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1134.eqiad.wmnet with reason: Maintenance [production]
09:46 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2167:3311', diff saved to https://phabricator.wikimedia.org/P39913 and previous config saved to /var/cache/conftool/dbconfig/20221116-094618-ladsgroup.json [production]
09:31 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2167:3311 (T318605)', diff saved to https://phabricator.wikimedia.org/P39912 and previous config saved to /var/cache/conftool/dbconfig/20221116-093112-ladsgroup.json [production]
09:18 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 293 [production]
09:17 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'configure' for AS: 293 [production]
09:16 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 30844 [production]
09:16 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'email' for AS: 30844 [production]
09:13 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 45899 [production]
09:13 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'email' for AS: 45899 [production]
08:47 <jmm@cumin2002> END (FAIL) - Cookbook sre.ganeti.addnode (exit_code=99) for new host ganeti1022.eqiad.wmnet to cluster eqiad and group D [production]
08:45 <jmm@cumin2002> START - Cookbook sre.ganeti.addnode for new host ganeti1022.eqiad.wmnet to cluster eqiad and group D [production]
08:45 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1022.eqiad.wmnet [production]
08:37 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1133.eqiad.wmnet with reason: Maintenance [production]
08:37 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1133.eqiad.wmnet with reason: Maintenance [production]
08:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1132 (T318605)', diff saved to https://phabricator.wikimedia.org/P39911 and previous config saved to /var/cache/conftool/dbconfig/20221116-083723-ladsgroup.json [production]
08:36 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti1022.eqiad.wmnet [production]
08:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1132', diff saved to https://phabricator.wikimedia.org/P39910 and previous config saved to /var/cache/conftool/dbconfig/20221116-082217-ladsgroup.json [production]
08:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1132', diff saved to https://phabricator.wikimedia.org/P39909 and previous config saved to /var/cache/conftool/dbconfig/20221116-080710-ladsgroup.json [production]
08:00 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2167:3311 (T318605)', diff saved to https://phabricator.wikimedia.org/P39908 and previous config saved to /var/cache/conftool/dbconfig/20221116-080001-ladsgroup.json [production]
07:59 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2167.codfw.wmnet with reason: Maintenance [production]
07:59 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2167.codfw.wmnet with reason: Maintenance [production]
07:59 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2153 (T318605)', diff saved to https://phabricator.wikimedia.org/P39907 and previous config saved to /var/cache/conftool/dbconfig/20221116-075940-ladsgroup.json [production]
07:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1132 (T318605)', diff saved to https://phabricator.wikimedia.org/P39906 and previous config saved to /var/cache/conftool/dbconfig/20221116-075204-ladsgroup.json [production]
07:44 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2153', diff saved to https://phabricator.wikimedia.org/P39905 and previous config saved to /var/cache/conftool/dbconfig/20221116-074433-ladsgroup.json [production]
07:29 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2153', diff saved to https://phabricator.wikimedia.org/P39904 and previous config saved to /var/cache/conftool/dbconfig/20221116-072926-ladsgroup.json [production]
07:14 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2153 (T318605)', diff saved to https://phabricator.wikimedia.org/P39903 and previous config saved to /var/cache/conftool/dbconfig/20221116-071420-ladsgroup.json [production]
06:36 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2127.codfw.wmnet with reason: Maintenance [production]
06:36 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db2127.codfw.wmnet with reason: Maintenance [production]
06:35 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1122.eqiad.wmnet with reason: Maintenance [production]
06:35 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1122.eqiad.wmnet with reason: Maintenance [production]
06:34 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2107.codfw.wmnet with reason: Maintenance [production]
06:34 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db2107.codfw.wmnet with reason: Maintenance [production]
06:23 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1132 (T318605)', diff saved to https://phabricator.wikimedia.org/P39902 and previous config saved to /var/cache/conftool/dbconfig/20221116-062315-ladsgroup.json [production]
06:23 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1132.eqiad.wmnet with reason: Maintenance [production]
06:22 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1132.eqiad.wmnet with reason: Maintenance [production]
06:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1128 (T318605)', diff saved to https://phabricator.wikimedia.org/P39901 and previous config saved to /var/cache/conftool/dbconfig/20221116-062253-ladsgroup.json [production]
06:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1128', diff saved to https://phabricator.wikimedia.org/P39900 and previous config saved to /var/cache/conftool/dbconfig/20221116-060747-ladsgroup.json [production]
05:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1128', diff saved to https://phabricator.wikimedia.org/P39899 and previous config saved to /var/cache/conftool/dbconfig/20221116-055241-ladsgroup.json [production]
05:49 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2153 (T318605)', diff saved to https://phabricator.wikimedia.org/P39898 and previous config saved to /var/cache/conftool/dbconfig/20221116-054935-ladsgroup.json [production]