1651-1700 of 10000 results (39ms)
2021-02-22 §
07:54 <marostegui@cumin1001> dbctl commit (dc=all): 'Remove db1090* from dbctl T274333', diff saved to https://phabricator.wikimedia.org/P14426 and previous config saved to /var/cache/conftool/dbconfig/20210222-075437-marostegui.json [production]
07:38 <moritzm> installing openldap security updates on LDAP replicas [production]
07:29 <hashar> Restarting CI Jenkins to downgrade plugin # T271683 [production]
07:14 <hashar> Restarting CI Jenkins for plugin upgrade # T271683 [production]
07:11 <elukey> powercycle elastic2045 - com2 available, no ssh, no root login (hangs indefinitely), no prometheus metrics reported [production]
2021-02-21 §
16:02 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1162 - crashed', diff saved to https://phabricator.wikimedia.org/P14424 and previous config saved to /var/cache/conftool/dbconfig/20210221-160258-marostegui.json [production]
10:07 <ariel@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on snapshot1008.eqiad.wmnet with reason: REIMAGE [production]
10:05 <ariel@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on snapshot1008.eqiad.wmnet with reason: REIMAGE [production]
09:32 <ariel@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on snapshot1008.eqiad.wmnet with reason: REIMAGE [production]
09:30 <ariel@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on snapshot1008.eqiad.wmnet with reason: REIMAGE [production]
09:29 <ariel@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dumpsdata1002.eqiad.wmnet [production]
09:23 <ariel@cumin1001> START - Cookbook sre.hosts.reboot-single for host dumpsdata1002.eqiad.wmnet [production]
2021-02-20 §
00:17 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1317.eqiad.wmnet [production]
00:16 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1317.eqiad.wmnet [production]
00:15 <ebernhardson> start batch processing images through MachineVision fetchSuggestions.php for T274220 on mwmaint1002 [production]
00:15 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1333.eqiad.wmnet [production]
00:13 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1333.eqiad.wmnet [production]
00:13 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1339.eqiad.wmnet [production]
00:13 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1342.eqiad.wmnet [production]
00:11 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1342.eqiad.wmnet [production]
2021-02-19 §
23:09 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1339.eqiad.wmnet [production]
23:05 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1317.eqiad.wmnet with reason: REIMAGE [production]
23:03 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1317.eqiad.wmnet with reason: REIMAGE [production]
22:49 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1342.eqiad.wmnet with reason: REIMAGE [production]
22:47 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1342.eqiad.wmnet with reason: REIMAGE [production]
22:44 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1333.eqiad.wmnet with reason: REIMAGE [production]
22:42 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1333.eqiad.wmnet with reason: REIMAGE [production]
22:33 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1340.eqiad.wmnet [production]
22:26 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1339.eqiad.wmnet with reason: REIMAGE [production]
22:24 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1339.eqiad.wmnet with reason: REIMAGE [production]
22:18 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1340.eqiad.wmnet [production]
22:16 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1320.eqiad.wmnet [production]
22:14 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1320.eqiad.wmnet [production]
22:10 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1262.eqiad.wmnet [production]
22:10 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1261.eqiad.wmnet [production]
22:09 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1261.eqiad.wmnet [production]
22:08 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1261.eqiad.wmnet [production]
21:50 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw2336.codfw.wmnet [production]
21:44 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw2336.codfw.wmnet [production]
21:34 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1340.eqiad.wmnet with reason: REIMAGE [production]
21:32 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1340.eqiad.wmnet with reason: REIMAGE [production]
21:27 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1320.eqiad.wmnet with reason: REIMAGE [production]
21:24 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1262.eqiad.wmnet with reason: REIMAGE [production]
21:24 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1320.eqiad.wmnet with reason: REIMAGE [production]
21:22 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2336.codfw.wmnet with reason: REIMAGE [production]
21:22 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1262.eqiad.wmnet with reason: REIMAGE [production]
21:20 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw2336.codfw.wmnet with reason: REIMAGE [production]
20:59 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1287.eqiad.wmnet [production]
20:57 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1287.eqiad.wmnet [production]
20:51 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw2257.codfw.wmnet [production]