351-400 of 10000 results (77ms)
2023-08-31 ยง
12:20 <cmooney@cumin1001> END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) [production]
12:19 <cmooney@cumin1001> END (ERROR) - Cookbook sre.dns.netbox (exit_code=97) [production]
12:18 <cmooney@cumin1001> END (FAIL) - Cookbook sre.network.provision (exit_code=93) for device lsw1-b6-codfw.mgmt.codfw.wmnet [production]
12:18 <cmooney@cumin1001> END (FAIL) - Cookbook sre.network.provision (exit_code=99) for device lsw1-b3-codfw.mgmt.codfw.wmnet [production]
12:17 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
12:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2158 (T343718)', diff saved to https://phabricator.wikimedia.org/P52197 and previous config saved to /var/cache/conftool/dbconfig/20230831-121721-ladsgroup.json [production]
12:17 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-b3-codfw.mgmt.codfw.wmnet [production]
12:17 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
12:17 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2187.codfw.wmnet with reason: Maintenance [production]
12:17 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
12:17 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-b4-codfw.mgmt.codfw.wmnet [production]
12:17 <cmooney@cumin1001> END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) [production]
12:17 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-b6-codfw.mgmt.codfw.wmnet [production]
12:17 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-b5-codfw.mgmt.codfw.wmnet [production]
12:17 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db2187.codfw.wmnet with reason: Maintenance [production]
12:17 <cmooney@cumin1001> END (FAIL) - Cookbook sre.network.provision (exit_code=99) for device lsw1-a6-codfw.mgmt.codfw.wmnet [production]
12:17 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2158.codfw.wmnet with reason: Maintenance [production]
12:17 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2158.codfw.wmnet with reason: Maintenance [production]
12:16 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2151 (T343718)', diff saved to https://phabricator.wikimedia.org/P52196 and previous config saved to /var/cache/conftool/dbconfig/20230831-121654-ladsgroup.json [production]
12:16 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-a6-codfw.mgmt.codfw.wmnet [production]
12:16 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
12:16 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-a5-codfw.mgmt.codfw.wmnet [production]
12:16 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
12:16 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-a4-codfw.mgmt.codfw.wmnet [production]
12:16 <cmooney@cumin1001> END (FAIL) - Cookbook sre.network.provision (exit_code=99) for device lsw1-a1-codfw.mgmt.codfw.wmnet [production]
12:16 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-a1-codfw.mgmt.codfw.wmnet [production]
12:15 <aqu@deploy1002> Finished deploy [analytics/refinery@06203c0]: Regular analytics weekly train [analytics/refinery@06203c0] (duration: 12m 15s) [production]
12:15 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
12:15 <cmooney@cumin1001> START - Cookbook sre.network.provision for device ssw1-a1-codfw.mgmt.codfw.wmnet [production]
12:11 <cmooney@cumin1001> END (FAIL) - Cookbook sre.network.provision (exit_code=99) for device lsw1-b2-codfw.mgmt.codfw.wmnet [production]
12:11 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
12:11 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-b2-codfw.mgmt.codfw.wmnet [production]
12:10 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-a8-codfw.mgmt.codfw.wmnet [production]
12:10 <cmooney@cumin1001> END (FAIL) - Cookbook sre.network.provision (exit_code=99) for device lsw1-a5-codfw.mgmt.codfw.wmnet [production]
12:10 <cmooney@cumin1001> END (FAIL) - Cookbook sre.network.provision (exit_code=99) for device lsw1-a6-codfw.mgmt.codfw.wmnet [production]
12:10 <cmooney@cumin1001> END (FAIL) - Cookbook sre.network.provision (exit_code=99) for device lsw1-a7-codfw.mgmt.codfw.wmnet [production]
12:10 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-a6-codfw.mgmt.codfw.wmnet [production]
12:10 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-a7-codfw.mgmt.codfw.wmnet [production]
12:10 <cmooney@cumin1001> END (FAIL) - Cookbook sre.network.provision (exit_code=99) for device lsw1-a4-codfw.mgmt.codfw.wmnet [production]
12:10 <cmooney@cumin1001> END (FAIL) - Cookbook sre.network.provision (exit_code=99) for device lsw1-a3-codfw.mgmt.codfw.wmnet [production]
12:10 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-a5-codfw.mgmt.codfw.wmnet [production]
12:10 <cmooney@cumin1001> END (FAIL) - Cookbook sre.network.provision (exit_code=99) for device lsw1-a2-codfw.mgmt.codfw.wmnet [production]
12:09 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-a4-codfw.mgmt.codfw.wmnet [production]
12:09 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-a3-codfw.mgmt.codfw.wmnet [production]
12:09 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-a2-codfw.mgmt.codfw.wmnet [production]
12:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1201', diff saved to https://phabricator.wikimedia.org/P52195 and previous config saved to /var/cache/conftool/dbconfig/20230831-120935-ladsgroup.json [production]
12:09 <cmooney@cumin1001> END (FAIL) - Cookbook sre.network.provision (exit_code=99) for device lsw1-a1-codfw.mgmt.codfw.wmnet [production]
12:09 <cmooney@cumin1001> START - Cookbook sre.network.provision for device lsw1-a1-codfw.mgmt.codfw.wmnet [production]
12:05 <ladsgroup@deploy1002> isaranto and ladsgroup: Backport for [[gerrit:953973|ores-extension: enable lift wing for fiwiki and itwiki (T343308)]] synced to the testservers mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option) [production]
12:03 <ladsgroup@deploy1002> Started scap: Backport for [[gerrit:953973|ores-extension: enable lift wing for fiwiki and itwiki (T343308)]] [production]