3301-3350 of 10000 results (50ms)
2022-03-08 ยง
21:29 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1156.eqiad.wmnet with reason: Maintenance [production]
21:29 <marostegui@cumin2002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1156.eqiad.wmnet with reason: Maintenance [production]
21:29 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1146:3312 (T298294)', diff saved to https://phabricator.wikimedia.org/P22145 and previous config saved to /var/cache/conftool/dbconfig/20220308-212939-marostegui.json [production]
21:28 <urbanecm@deploy1002> Synchronized php-1.38.0-wmf.25/extensions/DiscussionTools/includes/ApiDiscussionToolsEdit.php: cc5acc27ad02f473b001d088dba454e013129cb2: Fix handling of disabled mobileformat (T303262) (duration: 00m 49s) [production]
21:26 <urbanecm@deploy1002> Synchronized php-1.38.0-wmf.25/extensions/VisualEditor/includes/ApiVisualEditorEdit.php: a5c6d06714d76d84fac270e3dac4a4a0a2d83927: Fix handling of disabled mobileformat (T303262) (duration: 00m 49s) [production]
21:25 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:20 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:18 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1016.eqiad.wmnet with OS bullseye [production]
21:14 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:14 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:14 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1146:3312', diff saved to https://phabricator.wikimedia.org/P22144 and previous config saved to /var/cache/conftool/dbconfig/20220308-211404-marostegui.json [production]
21:10 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 3132fca7b3b078155fa406339d05286ca6e0797b: Enable DiscussionTools autotopicsub on MediaWiki.org (T302256) (duration: 00m 49s) [production]
21:07 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:03 <dancy@deploy1002> Pruned MediaWiki: 1.38.0-wmf.22 (duration: 01m 28s) [production]
21:01 <dancy@deploy1002> Pruned MediaWiki: 1.38.0-wmf.23 (duration: 01m 46s) [production]
20:59 <dancy@deploy1002> Finished scap: testwikis wikis to 1.38.0-wmf.25 refs T300201 (duration: 32m 13s) [production]
20:58 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1146:3312', diff saved to https://phabricator.wikimedia.org/P22143 and previous config saved to /var/cache/conftool/dbconfig/20220308-205829-marostegui.json [production]
20:42 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1146:3312 (T298294)', diff saved to https://phabricator.wikimedia.org/P22142 and previous config saved to /var/cache/conftool/dbconfig/20220308-204254-marostegui.json [production]
20:36 <rzl> rzl@apt1001:~$ sudo -i reprepro copy bullseye-wikimedia buster-wikimedia envoyproxy # T300324 [production]
20:36 <rzl> rzl@apt1001:~$ sudo -i reprepro copy stretch-wikimedia buster-wikimedia envoyproxy # T300324 [production]
20:27 <dancy@deploy1002> Started scap: testwikis wikis to 1.38.0-wmf.25 refs T300201 [production]
20:21 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1021.eqiad.wmnet with OS bullseye [production]
19:53 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1021.eqiad.wmnet with OS bullseye [production]
19:52 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1016.eqiad.wmnet with OS bullseye [production]
19:45 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1016.eqiad.wmnet with OS bullseye [production]
19:43 <XioNoX> !log push DHCP term to labs-in filters on eqiad cr [production]
19:42 <marostegui@cumin2002> dbctl commit (dc=all): 'Depooling db1146:3312 (T298294)', diff saved to https://phabricator.wikimedia.org/P22139 and previous config saved to /var/cache/conftool/dbconfig/20220308-194159-marostegui.json [production]
19:41 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1146.eqiad.wmnet with reason: Maintenance [production]
19:41 <marostegui@cumin2002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1146.eqiad.wmnet with reason: Maintenance [production]
19:39 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
19:39 <marostegui@cumin2002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
19:39 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1129 (T298294)', diff saved to https://phabricator.wikimedia.org/P22138 and previous config saved to /var/cache/conftool/dbconfig/20220308-193930-marostegui.json [production]
19:36 <cstone> updated donorwiki revision changed from 73de4731 to ca37a93e [production]
19:32 <dancy@deploy1002> Started scap: testwikis wikis to 1.38.0-wmf.25 refs T300201 [production]
19:27 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:23 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P22137 and previous config saved to /var/cache/conftool/dbconfig/20220308-192354-marostegui.json [production]
19:21 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
19:19 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:08 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P22136 and previous config saved to /var/cache/conftool/dbconfig/20220308-190818-marostegui.json [production]
18:55 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
18:53 <ejegg> updated payments-wiki from 3dfac3b2 to ca37a93e [production]
18:52 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1129 (T298294)', diff saved to https://phabricator.wikimedia.org/P22135 and previous config saved to /var/cache/conftool/dbconfig/20220308-185242-marostegui.json [production]
18:50 <marostegui@cumin2002> dbctl commit (dc=all): 'Depooling db1129 (T298294)', diff saved to https://phabricator.wikimedia.org/P22134 and previous config saved to /var/cache/conftool/dbconfig/20220308-185033-marostegui.json [production]
18:49 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
18:49 <marostegui@cumin2002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
18:49 <vgutierrez@puppetmaster1001> conftool action : set/pooled=inactive; selector: name=cp5004.eqsin.wmnet [production]
18:49 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1021.eqiad.wmnet with OS bullseye [production]
18:48 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
18:48 <marostegui@cumin2002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
18:47 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 16:00:00 on 8 hosts with reason: Maintenance [production]