|
2022-03-08
ยง
|
| 21:18 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudvirt1016.eqiad.wmnet with OS bullseye |
[production] |
| 21:14 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 21:14 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 21:14 |
<marostegui@cumin2002> |
dbctl commit (dc=all): 'Repooling after maintenance db1146:3312', diff saved to https://phabricator.wikimedia.org/P22144 and previous config saved to /var/cache/conftool/dbconfig/20220308-211404-marostegui.json |
[production] |
| 21:10 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 3132fca7b3b078155fa406339d05286ca6e0797b: Enable DiscussionTools autotopicsub on MediaWiki.org (T302256) (duration: 00m 49s) |
[production] |
| 21:07 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 21:03 |
<dancy@deploy1002> |
Pruned MediaWiki: 1.38.0-wmf.22 (duration: 01m 28s) |
[production] |
| 21:01 |
<dancy@deploy1002> |
Pruned MediaWiki: 1.38.0-wmf.23 (duration: 01m 46s) |
[production] |
| 20:59 |
<dancy@deploy1002> |
Finished scap: testwikis wikis to 1.38.0-wmf.25 refs T300201 (duration: 32m 13s) |
[production] |
| 20:58 |
<marostegui@cumin2002> |
dbctl commit (dc=all): 'Repooling after maintenance db1146:3312', diff saved to https://phabricator.wikimedia.org/P22143 and previous config saved to /var/cache/conftool/dbconfig/20220308-205829-marostegui.json |
[production] |
| 20:42 |
<marostegui@cumin2002> |
dbctl commit (dc=all): 'Repooling after maintenance db1146:3312 (T298294)', diff saved to https://phabricator.wikimedia.org/P22142 and previous config saved to /var/cache/conftool/dbconfig/20220308-204254-marostegui.json |
[production] |
| 20:36 |
<rzl> |
rzl@apt1001:~$ sudo -i reprepro copy bullseye-wikimedia buster-wikimedia envoyproxy # T300324 |
[production] |
| 20:36 |
<rzl> |
rzl@apt1001:~$ sudo -i reprepro copy stretch-wikimedia buster-wikimedia envoyproxy # T300324 |
[production] |
| 20:27 |
<dancy@deploy1002> |
Started scap: testwikis wikis to 1.38.0-wmf.25 refs T300201 |
[production] |
| 20:21 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1021.eqiad.wmnet with OS bullseye |
[production] |
| 19:53 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudvirt1021.eqiad.wmnet with OS bullseye |
[production] |
| 19:52 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1016.eqiad.wmnet with OS bullseye |
[production] |
| 19:45 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudvirt1016.eqiad.wmnet with OS bullseye |
[production] |
| 19:43 |
<XioNoX> |
!log push DHCP term to labs-in filters on eqiad cr |
[production] |
| 19:42 |
<marostegui@cumin2002> |
dbctl commit (dc=all): 'Depooling db1146:3312 (T298294)', diff saved to https://phabricator.wikimedia.org/P22139 and previous config saved to /var/cache/conftool/dbconfig/20220308-194159-marostegui.json |
[production] |
| 19:41 |
<marostegui@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1146.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:41 |
<marostegui@cumin2002> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1146.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:39 |
<marostegui@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1139.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:39 |
<marostegui@cumin2002> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1139.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:39 |
<marostegui@cumin2002> |
dbctl commit (dc=all): 'Repooling after maintenance db1129 (T298294)', diff saved to https://phabricator.wikimedia.org/P22138 and previous config saved to /var/cache/conftool/dbconfig/20220308-193930-marostegui.json |
[production] |
| 19:36 |
<cstone> |
updated donorwiki revision changed from 73de4731 to ca37a93e |
[production] |
| 19:32 |
<dancy@deploy1002> |
Started scap: testwikis wikis to 1.38.0-wmf.25 refs T300201 |
[production] |
| 19:27 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 19:23 |
<marostegui@cumin2002> |
dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P22137 and previous config saved to /var/cache/conftool/dbconfig/20220308-192354-marostegui.json |
[production] |
| 19:21 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 19:19 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 19:08 |
<marostegui@cumin2002> |
dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P22136 and previous config saved to /var/cache/conftool/dbconfig/20220308-190818-marostegui.json |
[production] |
| 18:55 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 18:53 |
<ejegg> |
updated payments-wiki from 3dfac3b2 to ca37a93e |
[production] |
| 18:52 |
<marostegui@cumin2002> |
dbctl commit (dc=all): 'Repooling after maintenance db1129 (T298294)', diff saved to https://phabricator.wikimedia.org/P22135 and previous config saved to /var/cache/conftool/dbconfig/20220308-185242-marostegui.json |
[production] |
| 18:50 |
<marostegui@cumin2002> |
dbctl commit (dc=all): 'Depooling db1129 (T298294)', diff saved to https://phabricator.wikimedia.org/P22134 and previous config saved to /var/cache/conftool/dbconfig/20220308-185033-marostegui.json |
[production] |
| 18:49 |
<marostegui@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1129.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:49 |
<marostegui@cumin2002> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1129.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:49 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=inactive; selector: name=cp5004.eqsin.wmnet |
[production] |
| 18:49 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1021.eqiad.wmnet with OS bullseye |
[production] |
| 18:48 |
<marostegui@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1102.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:48 |
<marostegui@cumin2002> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1102.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:47 |
<marostegui@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 16:00:00 on 8 hosts with reason: Maintenance |
[production] |
| 18:47 |
<marostegui@cumin2002> |
START - Cookbook sre.hosts.downtime for 16:00:00 on 8 hosts with reason: Maintenance |
[production] |
| 18:47 |
<marostegui@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2104.codfw.wmnet with reason: Maintenance |
[production] |
| 18:47 |
<marostegui@cumin2002> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db2104.codfw.wmnet with reason: Maintenance |
[production] |
| 18:47 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=inactive; selector: name=cp1085.eqiad.wmnet |
[production] |
| 18:44 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 18:35 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 18:35 |
<bblack> |
cp10[3579] - restarting varnish-fe |
[production] |