3701-3750 of 10000 results (35ms)
2021-02-16 ยง
20:04 <legoktm@cumin1001> conftool action : set/pooled=no; selector: name=mw1290.eqiad.wmnet [production]
20:04 <legoktm@cumin1001> conftool action : set/pooled=no; selector: name=mw1289.eqiad.wmnet [production]
20:03 <legoktm@cumin1001> conftool action : set/pooled=no; selector: name=mw1288.eqiad.wmnet [production]
19:58 <ryankemper> [WDQS] De-pooled `wdqs100[4,7]` to catch up on lag, and pooled `wdqs100[5,6]` [production]
19:09 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3:00:00 on mwdebug1002.eqiad.wmnet with reason: OS upgrade [production]
19:09 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 3:00:00 on mwdebug1002.eqiad.wmnet with reason: OS upgrade [production]
19:06 <legoktm@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1297.eqiad.wmnet with reason: REIMAGE [production]
19:04 <legoktm@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1290.eqiad.wmnet with reason: REIMAGE [production]
19:03 <legoktm@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1297.eqiad.wmnet with reason: REIMAGE [production]
19:02 <legoktm@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1289.eqiad.wmnet with reason: REIMAGE [production]
19:01 <legoktm@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1290.eqiad.wmnet with reason: REIMAGE [production]
19:00 <legoktm@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1288.eqiad.wmnet with reason: REIMAGE [production]
18:59 <mutante> puppetmaster1002 - puppet cert clean mwdebug1002.eqiad.wmnet, sign new request, initial puppet run (T274023) [production]
18:59 <legoktm@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1289.eqiad.wmnet with reason: REIMAGE [production]
18:58 <legoktm@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1288.eqiad.wmnet with reason: REIMAGE [production]
18:52 <mutante> re-creating mwdebug1002 [production]
18:49 <dduvall@deploy1001> Finished scap: testwikis wikis to 1.36.0-wmf.31 (duration: 49m 37s) [production]
18:41 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1346.eqiad.wmnet [production]
18:38 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1352.eqiad.wmnet [production]
18:37 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1347.eqiad.wmnet [production]
18:35 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1346.eqiad.wmnet [production]
18:33 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1352.eqiad.wmnet [production]
18:32 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1347.eqiad.wmnet [production]
18:28 <mutante> mw1352 - powercycle via mgmt [production]
18:04 <dduvall@deploy1001> Started scap: testwikis wikis to 1.36.0-wmf.31 [production]
17:41 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1346.eqiad.wmnet with reason: REIMAGE [production]
17:39 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1347.eqiad.wmnet with reason: REIMAGE [production]
17:39 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1346.eqiad.wmnet with reason: REIMAGE [production]
17:37 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1347.eqiad.wmnet with reason: REIMAGE [production]
17:36 <marxarelli> 1.36.0-wmf.31 was branched at c49ac6d2448efa085bdd34fc415aeece05a98dde (T271345) [production]
17:33 <akosiaris@deploy1001> helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. [production]
17:32 <akosiaris@deploy1001> helmfile [staging-codfw] START helmfile.d/admin 'sync'. [production]
17:31 <akosiaris@deploy1001> helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. [production]
17:30 <akosiaris@deploy1001> helmfile [staging-codfw] START helmfile.d/admin 'sync'. [production]
17:30 <akosiaris@deploy1001> helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. [production]
17:30 <akosiaris@deploy1001> helmfile [staging-codfw] START helmfile.d/admin 'sync'. [production]
17:29 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1352.eqiad.wmnet with reason: REIMAGE [production]
17:27 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1352.eqiad.wmnet with reason: REIMAGE [production]
17:24 <godog> swift codfw-prod decrease HDD weight for ms-be20[16-27] - T272837 [production]
17:23 <jforrester@deploy1001> Finished deploy [integration/docroot@8ab9125]: Update docroot with Special:MyLanguage links. (duration: 00m 11s) [production]
17:23 <jforrester@deploy1001> Started deploy [integration/docroot@8ab9125]: Update docroot with Special:MyLanguage links. [production]
17:21 <akosiaris@deploy1001> helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. [production]
17:21 <akosiaris@deploy1001> helmfile [staging-codfw] START helmfile.d/admin 'sync'. [production]
17:18 <akosiaris@deploy1001> helmfile [staging-codfw] START helmfile.d/admin 'sync'. [production]
16:25 <klausman@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ml-etcd[2001-2003].codfw.wmnet with reason: klausman: Pushing new etcd changes from T273071 [production]
16:25 <klausman@cumin2001> START - Cookbook sre.hosts.downtime for 2:00:00 on ml-etcd[2001-2003].codfw.wmnet with reason: klausman: Pushing new etcd changes from T273071 [production]
16:17 <moritzm> installing edk2 security updates [production]
16:09 <moritzm> installing python-bottle security updates on buster [production]
15:58 <papaul> power down ms-be2031 for firmware upgrade [production]
15:44 <klausman@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ml-etcd[1001-1003].eqiad.wmnet with reason: klausman: Pushing new etcd changes from T273071 [production]