|
2022-11-16
ยง
|
| 13:55 |
<Emperor> |
set thanos ring replicas to 3.20 T311690 |
[production] |
| 13:45 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1135 (T318605)', diff saved to https://phabricator.wikimedia.org/P39929 and previous config saved to /var/cache/conftool/dbconfig/20221116-134543-ladsgroup.json |
[production] |
| 13:25 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2105 (T323214)', diff saved to https://phabricator.wikimedia.org/P39928 and previous config saved to /var/cache/conftool/dbconfig/20221116-132531-ladsgroup.json |
[production] |
| 13:25 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2105.codfw.wmnet with reason: Maintenance |
[production] |
| 13:25 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2105.codfw.wmnet with reason: Maintenance |
[production] |
| 12:54 |
<aborrero@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudgw1002.eqiad.wmnet with OS bullseye |
[production] |
| 12:49 |
<sukhe@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp2042.codfw.wmnet with OS bullseye |
[production] |
| 12:39 |
<aborrero@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudgw1002.eqiad.wmnet with reason: host reimage |
[production] |
| 12:19 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2170:3311', diff saved to https://phabricator.wikimedia.org/P39925 and previous config saved to /var/cache/conftool/dbconfig/20221116-121934-ladsgroup.json |
[production] |
| 12:17 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1135 (T318605)', diff saved to https://phabricator.wikimedia.org/P39924 and previous config saved to /var/cache/conftool/dbconfig/20221116-121701-ladsgroup.json |
[production] |
| 12:16 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1135.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:16 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1135.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:16 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1134 (T318605)', diff saved to https://phabricator.wikimedia.org/P39923 and previous config saved to /var/cache/conftool/dbconfig/20221116-121628-ladsgroup.json |
[production] |
| 12:07 |
<sukhe@cumin2002> |
START - Cookbook sre.hosts.reimage for host cp2042.codfw.wmnet with OS bullseye |
[production] |
| 12:04 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2170:3311', diff saved to https://phabricator.wikimedia.org/P39922 and previous config saved to /var/cache/conftool/dbconfig/20221116-120428-ladsgroup.json |
[production] |
| 12:01 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1134', diff saved to https://phabricator.wikimedia.org/P39921 and previous config saved to /var/cache/conftool/dbconfig/20221116-120122-ladsgroup.json |
[production] |
| 11:49 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2170:3311 (T318605)', diff saved to https://phabricator.wikimedia.org/P39920 and previous config saved to /var/cache/conftool/dbconfig/20221116-114921-ladsgroup.json |
[production] |
| 11:46 |
<aborrero@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudgw1001.eqiad.wmnet with OS bullseye |
[production] |
| 11:46 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1134', diff saved to https://phabricator.wikimedia.org/P39919 and previous config saved to /var/cache/conftool/dbconfig/20221116-114615-ladsgroup.json |
[production] |
| 11:31 |
<aborrero@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudgw1001.eqiad.wmnet with reason: host reimage |
[production] |
| 11:31 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1134 (T318605)', diff saved to https://phabricator.wikimedia.org/P39918 and previous config saved to /var/cache/conftool/dbconfig/20221116-113108-ladsgroup.json |
[production] |
| 11:29 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2127.codfw.wmnet with reason: Maintenance |
[production] |
| 11:29 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db2127.codfw.wmnet with reason: Maintenance |
[production] |
| 11:27 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1123.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:27 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1123.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:26 |
<aborrero@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudgw1001.eqiad.wmnet with reason: host reimage |
[production] |
| 11:14 |
<aborrero@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudgw1001.eqiad.wmnet with OS bullseye |
[production] |
| 10:30 |
<urbanecm> |
Run `mwscript extensions/GrowthExperiments/maintenance/updateIsActiveFlagForMentees.php` for all wikis in growthexperiments.dblist at mwmaint1002 (T318457) |
[production] |
| 10:29 |
<jynus> |
restarting apache on lists.wm.o |
[production] |
| 10:16 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2170:3311 (T318605)', diff saved to https://phabricator.wikimedia.org/P39917 and previous config saved to /var/cache/conftool/dbconfig/20221116-101653-ladsgroup.json |
[production] |
| 10:16 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2170.codfw.wmnet with reason: Maintenance |
[production] |
| 10:16 |
<kevinbazira@deploy1002> |
Finished deploy [ores/deploy@0114799]: T319373 (duration: 10m 51s) |
[production] |
| 10:16 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2170.codfw.wmnet with reason: Maintenance |
[production] |
| 10:16 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2167:3311 (T318605)', diff saved to https://phabricator.wikimedia.org/P39916 and previous config saved to /var/cache/conftool/dbconfig/20221116-101631-ladsgroup.json |
[production] |
| 10:05 |
<kevinbazira@deploy1002> |
Started deploy [ores/deploy@0114799]: T319373 |
[production] |
| 10:01 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2167:3311', diff saved to https://phabricator.wikimedia.org/P39915 and previous config saved to /var/cache/conftool/dbconfig/20221116-100125-ladsgroup.json |
[production] |
| 10:00 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1134 (T318605)', diff saved to https://phabricator.wikimedia.org/P39914 and previous config saved to /var/cache/conftool/dbconfig/20221116-100027-ladsgroup.json |
[production] |
| 10:00 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1134.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:59 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1134.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:46 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2167:3311', diff saved to https://phabricator.wikimedia.org/P39913 and previous config saved to /var/cache/conftool/dbconfig/20221116-094618-ladsgroup.json |
[production] |
| 09:31 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2167:3311 (T318605)', diff saved to https://phabricator.wikimedia.org/P39912 and previous config saved to /var/cache/conftool/dbconfig/20221116-093112-ladsgroup.json |
[production] |
| 09:18 |
<ayounsi@cumin1001> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 293 |
[production] |
| 09:17 |
<ayounsi@cumin1001> |
START - Cookbook sre.network.peering with action 'configure' for AS: 293 |
[production] |
| 09:16 |
<ayounsi@cumin1001> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 30844 |
[production] |
| 09:16 |
<ayounsi@cumin1001> |
START - Cookbook sre.network.peering with action 'email' for AS: 30844 |
[production] |
| 09:13 |
<ayounsi@cumin1001> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 45899 |
[production] |
| 09:13 |
<ayounsi@cumin1001> |
START - Cookbook sre.network.peering with action 'email' for AS: 45899 |
[production] |
| 08:47 |
<jmm@cumin2002> |
END (FAIL) - Cookbook sre.ganeti.addnode (exit_code=99) for new host ganeti1022.eqiad.wmnet to cluster eqiad and group D |
[production] |
| 08:45 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.addnode for new host ganeti1022.eqiad.wmnet to cluster eqiad and group D |
[production] |
| 08:45 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1022.eqiad.wmnet |
[production] |