|
2022-11-17
ยง
|
| 12:13 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2026.codfw.wmnet |
[production] |
| 12:12 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.maps.roll-restart (exit_code=0) rolling restart_daemons on A:maps-replica-codfw |
[production] |
| 12:10 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P40101 and previous config saved to /var/cache/conftool/dbconfig/20221117-121026-ladsgroup.json |
[production] |
| 12:06 |
<Emperor> |
restart swift proxies to deploy phonos changes to rewrite.py T317417 |
[production] |
| 12:06 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2026.codfw.wmnet |
[production] |
| 12:02 |
<urbanecm> |
[urbanecm@mwmaint1002 ~]$ time mwscript extensions/GrowthExperiments/maintenance/updateIsActiveFlagForMentees.php --wiki=trwiki # T318457 |
[production] |
| 12:01 |
<hashar> |
Gerrit back since 11:45 UTC |
[production] |
| 12:01 |
<urbanecm> |
[urbanecm@mwmaint1002 ~]$ time mwscript extensions/GrowthExperiments/maintenance/updateIsActiveFlagForMentees.php --wiki=enwiki # T318457 |
[production] |
| 11:55 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P40100 and previous config saved to /var/cache/conftool/dbconfig/20221117-115520-ladsgroup.json |
[production] |
| 11:50 |
<jmm@cumin2002> |
START - Cookbook sre.maps.roll-restart rolling restart_daemons on A:maps-replica-codfw |
[production] |
| 11:48 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2025.codfw.wmnet |
[production] |
| 11:47 |
<sukhe@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=cp5032.eqsin.wmnet |
[production] |
| 11:47 |
<sukhe@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=cp5032.eqsin.wmnet |
[production] |
| 11:41 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2025.codfw.wmnet |
[production] |
| 11:40 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1166 (T318955)', diff saved to https://phabricator.wikimedia.org/P40099 and previous config saved to /var/cache/conftool/dbconfig/20221117-114013-ladsgroup.json |
[production] |
| 11:38 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2175 (T323214)', diff saved to https://phabricator.wikimedia.org/P40098 and previous config saved to /var/cache/conftool/dbconfig/20221117-113814-ladsgroup.json |
[production] |
| 11:36 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1166 (T318955)', diff saved to https://phabricator.wikimedia.org/P40097 and previous config saved to /var/cache/conftool/dbconfig/20221117-113621-ladsgroup.json |
[production] |
| 11:36 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1166.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:36 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1166.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:35 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2024.codfw.wmnet |
[production] |
| 11:27 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2024.codfw.wmnet |
[production] |
| 11:23 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2175', diff saved to https://phabricator.wikimedia.org/P40096 and previous config saved to /var/cache/conftool/dbconfig/20221117-112307-ladsgroup.json |
[production] |
| 11:20 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2022.codfw.wmnet |
[production] |
| 11:17 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1182 (T323214)', diff saved to https://phabricator.wikimedia.org/P40095 and previous config saved to /var/cache/conftool/dbconfig/20221117-111745-ladsgroup.json |
[production] |
| 11:17 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1182.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:17 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1182.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:17 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1170:3312 (T323214)', diff saved to https://phabricator.wikimedia.org/P40094 and previous config saved to /var/cache/conftool/dbconfig/20221117-111712-ladsgroup.json |
[production] |
| 11:13 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2022.codfw.wmnet |
[production] |
| 11:08 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2175', diff saved to https://phabricator.wikimedia.org/P40093 and previous config saved to /var/cache/conftool/dbconfig/20221117-110801-ladsgroup.json |
[production] |
| 11:02 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2021.codfw.wmnet |
[production] |
| 11:02 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1170:3312', diff saved to https://phabricator.wikimedia.org/P40092 and previous config saved to /var/cache/conftool/dbconfig/20221117-110206-ladsgroup.json |
[production] |
| 10:55 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2021.codfw.wmnet |
[production] |
| 10:52 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2175 (T323214)', diff saved to https://phabricator.wikimedia.org/P40091 and previous config saved to /var/cache/conftool/dbconfig/20221117-105254-ladsgroup.json |
[production] |
| 10:47 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1170:3312', diff saved to https://phabricator.wikimedia.org/P40090 and previous config saved to /var/cache/conftool/dbconfig/20221117-104659-ladsgroup.json |
[production] |
| 10:45 |
<moritzm> |
restarting apache/FPM on mw canaries to pick up gnutls security updates |
[production] |
| 10:31 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1170:3312 (T323214)', diff saved to https://phabricator.wikimedia.org/P40089 and previous config saved to /var/cache/conftool/dbconfig/20221117-103153-ladsgroup.json |
[production] |
| 10:25 |
<vgutierrez> |
pool ats-be@cp2042 |
[production] |
| 10:20 |
<moritzm> |
installing gnutls28 security updates on Buster |
[production] |
| 10:20 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2019.codfw.wmnet |
[production] |
| 10:19 |
<hashar> |
gerrit1001: removed 5G of 2019's thread dumps in `/srv/home-cobalt.wikimedia.org/thcipriani/threaddumps` |
[production] |
| 10:12 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2019.codfw.wmnet |
[production] |
| 10:03 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2018.codfw.wmnet |
[production] |
| 09:56 |
<hashar> |
Stopped Gerrit and running offline reindexing |
[production] |
| 09:55 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2018.codfw.wmnet |
[production] |
| 09:52 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2017.codfw.wmnet |
[production] |
| 09:45 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2017.codfw.wmnet |
[production] |
| 09:42 |
<hashar> |
Cleaning gerrit1001.wikimedia.org `/` partition |
[production] |
| 09:37 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2016.codfw.wmnet |
[production] |
| 09:36 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2175 (T323214)', diff saved to https://phabricator.wikimedia.org/P40087 and previous config saved to /var/cache/conftool/dbconfig/20221117-093650-ladsgroup.json |
[production] |
| 09:36 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2175.codfw.wmnet with reason: Maintenance |
[production] |