1201-1250 of 10000 results (45ms)
2022-03-08 ยง
19:42 <marostegui@cumin2002> dbctl commit (dc=all): 'Depooling db1146:3312 (T298294)', diff saved to https://phabricator.wikimedia.org/P22139 and previous config saved to /var/cache/conftool/dbconfig/20220308-194159-marostegui.json [production]
19:41 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1146.eqiad.wmnet with reason: Maintenance [production]
19:41 <marostegui@cumin2002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1146.eqiad.wmnet with reason: Maintenance [production]
19:39 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
19:39 <marostegui@cumin2002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
19:39 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1129 (T298294)', diff saved to https://phabricator.wikimedia.org/P22138 and previous config saved to /var/cache/conftool/dbconfig/20220308-193930-marostegui.json [production]
19:36 <cstone> updated donorwiki revision changed from 73de4731 to ca37a93e [production]
19:32 <dancy@deploy1002> Started scap: testwikis wikis to 1.38.0-wmf.25 refs T300201 [production]
19:27 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:23 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P22137 and previous config saved to /var/cache/conftool/dbconfig/20220308-192354-marostegui.json [production]
19:21 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
19:19 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:08 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P22136 and previous config saved to /var/cache/conftool/dbconfig/20220308-190818-marostegui.json [production]
18:55 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
18:53 <ejegg> updated payments-wiki from 3dfac3b2 to ca37a93e [production]
18:52 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1129 (T298294)', diff saved to https://phabricator.wikimedia.org/P22135 and previous config saved to /var/cache/conftool/dbconfig/20220308-185242-marostegui.json [production]
18:50 <marostegui@cumin2002> dbctl commit (dc=all): 'Depooling db1129 (T298294)', diff saved to https://phabricator.wikimedia.org/P22134 and previous config saved to /var/cache/conftool/dbconfig/20220308-185033-marostegui.json [production]
18:49 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
18:49 <marostegui@cumin2002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
18:49 <vgutierrez@puppetmaster1001> conftool action : set/pooled=inactive; selector: name=cp5004.eqsin.wmnet [production]
18:49 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1021.eqiad.wmnet with OS bullseye [production]
18:48 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
18:48 <marostegui@cumin2002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
18:47 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 16:00:00 on 8 hosts with reason: Maintenance [production]
18:47 <marostegui@cumin2002> START - Cookbook sre.hosts.downtime for 16:00:00 on 8 hosts with reason: Maintenance [production]
18:47 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2104.codfw.wmnet with reason: Maintenance [production]
18:47 <marostegui@cumin2002> START - Cookbook sre.hosts.downtime for 8:00:00 on db2104.codfw.wmnet with reason: Maintenance [production]
18:47 <vgutierrez@puppetmaster1001> conftool action : set/pooled=inactive; selector: name=cp1085.eqiad.wmnet [production]
18:44 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
18:35 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
18:35 <bblack> cp10[3579] - restarting varnish-fe [production]
18:29 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1021.eqiad.wmnet with OS bullseye [production]
18:27 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host restbase1032.eqiad.wmnet with OS buster [production]
18:21 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-fe1012.eqiad.wmnet with OS stretch [production]
18:14 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on restbase1032.eqiad.wmnet with reason: host reimage [production]
18:11 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-fe1012.eqiad.wmnet with reason: host reimage [production]
18:10 <cmjohnson@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on restbase1032.eqiad.wmnet with reason: host reimage [production]
18:07 <cmjohnson@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on ms-fe1012.eqiad.wmnet with reason: host reimage [production]
17:58 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host restbase1032.eqiad.wmnet with OS buster [production]
17:57 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host ms-fe1012.eqiad.wmnet with OS stretch [production]
17:48 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1161 (T298294)', diff saved to https://phabricator.wikimedia.org/P22133 and previous config saved to /var/cache/conftool/dbconfig/20220308-174838-marostegui.json [production]
17:33 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1017.eqiad.wmnet with OS bullseye [production]
17:33 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1161', diff saved to https://phabricator.wikimedia.org/P22132 and previous config saved to /var/cache/conftool/dbconfig/20220308-173302-marostegui.json [production]
17:27 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1017.eqiad.wmnet with OS bullseye [production]
17:17 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1161', diff saved to https://phabricator.wikimedia.org/P22131 and previous config saved to /var/cache/conftool/dbconfig/20220308-171728-marostegui.json [production]
17:07 <jbond> deploy minor clean up of puppetmaster classes gerrit:769072 [production]
17:01 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1161 (T298294)', diff saved to https://phabricator.wikimedia.org/P22130 and previous config saved to /var/cache/conftool/dbconfig/20220308-170153-marostegui.json [production]
17:01 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1016.eqiad.wmnet with OS bullseye [production]
16:58 <marostegui@cumin2002> dbctl commit (dc=all): 'Depooling db1161 (T298294)', diff saved to https://phabricator.wikimedia.org/P22129 and previous config saved to /var/cache/conftool/dbconfig/20220308-165843-marostegui.json [production]
16:58 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 16:00:00 on clouddb[1016,1020-1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]