|
2022-11-17
ยง
|
| 14:34 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2015.codfw.wmnet |
[production] |
| 14:34 |
<vgutierrez> |
depool cp2042 |
[production] |
| 14:33 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1197 (T323214)', diff saved to https://phabricator.wikimedia.org/P40113 and previous config saved to /var/cache/conftool/dbconfig/20221117-143334-ladsgroup.json |
[production] |
| 14:33 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1197.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:33 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1197.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:33 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1188 (T323214)', diff saved to https://phabricator.wikimedia.org/P40112 and previous config saved to /var/cache/conftool/dbconfig/20221117-143313-ladsgroup.json |
[production] |
| 14:31 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on ganeti1019.eqiad.wmnet with reason: Remove from cluster for eventual reimage |
[production] |
| 14:30 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on ganeti1019.eqiad.wmnet with reason: Remove from cluster for eventual reimage |
[production] |
| 14:29 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host mc-wf1001.eqiad.wmnet |
[production] |
| 14:25 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2015.codfw.wmnet |
[production] |
| 14:18 |
<urbanecm@deploy1002> |
Sync cancelled. |
[production] |
| 14:18 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1188', diff saved to https://phabricator.wikimedia.org/P40111 and previous config saved to /var/cache/conftool/dbconfig/20221117-141806-ladsgroup.json |
[production] |
| 14:14 |
<urbanecm@deploy1002> |
urbanecm and matmarex: Backport for [[gerrit:858308|Make "Add topic" button sticky (T316175)]], [[gerrit:858309|CommentFormatter: Fix condition for lede button to consider new wrappers (T323171)]], [[gerrit:858310|Remove override for Minerva hiding .tmbox, no longer needed (T257394)]], [[gerrit:858311|CommentFormatter: Fix condition for lede button to consider table of contents (T323241)]], [[gerr |
[production] |
| 14:13 |
<urbanecm@deploy1002> |
Started scap: Backport for [[gerrit:858308|Make "Add topic" button sticky (T316175)]], [[gerrit:858309|CommentFormatter: Fix condition for lede button to consider new wrappers (T323171)]], [[gerrit:858310|Remove override for Minerva hiding .tmbox, no longer needed (T257394)]], [[gerrit:858311|CommentFormatter: Fix condition for lede button to consider table of contents (T323241)]], [[gerrit:858312 |
[production] |
| 14:12 |
<urbanecm@deploy1002> |
Finished scap: Backport for [[gerrit:856705|fiwiktionary: Add rollbacker group (T323063)]] (duration: 06m 35s) |
[production] |
| 14:06 |
<urbanecm@deploy1002> |
urbanecm and stang: Backport for [[gerrit:856705|fiwiktionary: Add rollbacker group (T323063)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet |
[production] |
| 14:05 |
<urbanecm@deploy1002> |
Started scap: Backport for [[gerrit:856705|fiwiktionary: Add rollbacker group (T323063)]] |
[production] |
| 14:05 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2020.codfw.wmnet |
[production] |
| 14:03 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1188', diff saved to https://phabricator.wikimedia.org/P40110 and previous config saved to /var/cache/conftool/dbconfig/20221117-140300-ladsgroup.json |
[production] |
| 14:00 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2020.codfw.wmnet |
[production] |
| 13:58 |
<ayounsi@cumin1001> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 6774 |
[production] |
| 13:56 |
<ayounsi@cumin1001> |
START - Cookbook sre.network.peering with action 'configure' for AS: 6774 |
[production] |
| 13:52 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=maps2008.codfw.wmnet |
[production] |
| 13:47 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1188 (T323214)', diff saved to https://phabricator.wikimedia.org/P40109 and previous config saved to /var/cache/conftool/dbconfig/20221117-134753-ladsgroup.json |
[production] |
| 13:46 |
<moritzm> |
failover ganeti master in codfw to ganeti2021 |
[production] |
| 13:40 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2030.codfw.wmnet |
[production] |
| 13:32 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2030.codfw.wmnet |
[production] |
| 13:22 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2029.codfw.wmnet |
[production] |
| 13:17 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1188 (T323214)', diff saved to https://phabricator.wikimedia.org/P40108 and previous config saved to /var/cache/conftool/dbconfig/20221117-131709-ladsgroup.json |
[production] |
| 13:17 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1188.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:16 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1188.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:16 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1182 (T323214)', diff saved to https://phabricator.wikimedia.org/P40107 and previous config saved to /var/cache/conftool/dbconfig/20221117-131647-ladsgroup.json |
[production] |
| 13:14 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2029.codfw.wmnet |
[production] |
| 13:09 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2028.codfw.wmnet |
[production] |
| 13:01 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2028.codfw.wmnet |
[production] |
| 13:01 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1182', diff saved to https://phabricator.wikimedia.org/P40106 and previous config saved to /var/cache/conftool/dbconfig/20221117-130141-ladsgroup.json |
[production] |
| 12:55 |
<mfossati@deploy1002> |
Finished deploy [airflow-dags/platform_eng@4bdda20]: (no justification provided) (duration: 00m 18s) |
[production] |
| 12:55 |
<mfossati@deploy1002> |
Started deploy [airflow-dags/platform_eng@4bdda20]: (no justification provided) |
[production] |
| 12:46 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1182', diff saved to https://phabricator.wikimedia.org/P40105 and previous config saved to /var/cache/conftool/dbconfig/20221117-124634-ladsgroup.json |
[production] |
| 12:32 |
<mfossati@deploy1002> |
Finished deploy [airflow-dags/platform_eng@3bb99c2]: (no justification provided) (duration: 00m 05s) |
[production] |
| 12:32 |
<mfossati@deploy1002> |
Started deploy [airflow-dags/platform_eng@3bb99c2]: (no justification provided) |
[production] |
| 12:31 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1182 (T323214)', diff saved to https://phabricator.wikimedia.org/P40104 and previous config saved to /var/cache/conftool/dbconfig/20221117-123128-ladsgroup.json |
[production] |
| 12:30 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2027.codfw.wmnet |
[production] |
| 12:29 |
<moritzm> |
installing bluez security updates |
[production] |
| 12:25 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1166 (T318955)', diff saved to https://phabricator.wikimedia.org/P40103 and previous config saved to /var/cache/conftool/dbconfig/20221117-122532-ladsgroup.json |
[production] |
| 12:24 |
<moritzm> |
restarting slapd on serpens/seaborgium/ldap-corp* to pick up GNUTLS update |
[production] |
| 12:23 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2027.codfw.wmnet |
[production] |
| 12:22 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.maps.roll-restart (exit_code=0) rolling restart_daemons on A:maps-replica-eqiad |
[production] |
| 12:18 |
<jmm@cumin2002> |
START - Cookbook sre.maps.roll-restart rolling restart_daemons on A:maps-replica-eqiad |
[production] |
| 12:13 |
<sukhe> |
rolling restart of A:wikidough to pick up security updates |
[production] |