4351-4400 of 10000 results (40ms)
2021-02-11 §
01:05 <dwisehaupt> Move payments/civi/frpig into maint mode for civi upgrade [production]
01:04 <eileen> process-control config revision is f216d8fe8e [production]
00:26 <legoktm@deploy1001> Synchronized wmf-config/profiler.php: Revert "profiler: Send data to excimer-buster pipeline" (duration: 02m 00s) [production]
00:03 <milimetric@deploy1001> Finished deploy [analytics/refinery@3da19b6] (thin): More fixes for jobs after cluster upgrade (duration: 00m 07s) [production]
00:03 <milimetric@deploy1001> Started deploy [analytics/refinery@3da19b6] (thin): More fixes for jobs after cluster upgrade [production]
2021-02-10 §
23:53 <milimetric@deploy1001> Finished deploy [analytics/refinery@3da19b6]: More fixes for jobs after cluster upgrade (duration: 14m 23s) [production]
23:49 <legoktm@cumin1001> conftool action : set/pooled=yes; selector: name=mw1328.eqiad.wmnet [production]
23:49 <legoktm@cumin1001> conftool action : set/pooled=yes; selector: name=mw1327.eqiad.wmnet [production]
23:49 <legoktm@cumin1001> conftool action : set/pooled=yes; selector: name=mw1326.eqiad.wmnet [production]
23:49 <legoktm@cumin1001> conftool action : set/pooled=yes; selector: name=mw1325.eqiad.wmnet [production]
23:38 <milimetric@deploy1001> Started deploy [analytics/refinery@3da19b6]: More fixes for jobs after cluster upgrade [production]
23:36 <eileen> civicrm revision changed from ae24f87158 to 2ce8194c07, config revision is a48a7db0a2 [production]
22:37 <legoktm@cumin1001> conftool action : set/pooled=no; selector: name=mw1328.eqiad.wmnet [production]
22:37 <legoktm@cumin1001> conftool action : set/pooled=no; selector: name=mw1327.eqiad.wmnet [production]
22:37 <legoktm@cumin1001> conftool action : set/pooled=no; selector: name=mw1326.eqiad.wmnet [production]
22:37 <legoktm@cumin1001> conftool action : set/pooled=no; selector: name=mw1325.eqiad.wmnet [production]
22:32 <ebernhardson@deploy1001> Finished deploy [wikimedia/discovery/analytics@d97f7d9]: query_clicks: Remove result file merging (duration: 01m 27s) [production]
22:30 <ebernhardson@deploy1001> Started deploy [wikimedia/discovery/analytics@d97f7d9]: query_clicks: Remove result file merging [production]
22:24 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1377.eqiad.wmnet [production]
22:23 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1369.eqiad.wmnet [production]
22:14 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1377.eqiad.wmnet [production]
22:13 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1369.eqiad.wmnet [production]
22:07 <mutante> mw1369, mw1377 - all servers in this section now consistenly fail to reboot when triggered as the last step of wmf-reimage script [production]
21:43 <legoktm@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1328.eqiad.wmnet with reason: REIMAGE [production]
21:41 <legoktm@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1327.eqiad.wmnet with reason: REIMAGE [production]
21:41 <legoktm@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1328.eqiad.wmnet with reason: REIMAGE [production]
21:39 <legoktm@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1326.eqiad.wmnet with reason: REIMAGE [production]
21:39 <legoktm@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1327.eqiad.wmnet with reason: REIMAGE [production]
21:37 <legoktm@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1325.eqiad.wmnet with reason: REIMAGE [production]
21:37 <legoktm@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1326.eqiad.wmnet with reason: REIMAGE [production]
21:35 <legoktm@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1325.eqiad.wmnet with reason: REIMAGE [production]
21:06 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1377.eqiad.wmnet with reason: REIMAGE [production]
21:04 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1369.eqiad.wmnet with reason: REIMAGE [production]
21:02 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1377.eqiad.wmnet with reason: REIMAGE [production]
21:01 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1369.eqiad.wmnet with reason: REIMAGE [production]
20:39 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1293.eqiad.wmnet [production]
20:37 <eileen> civicrm revision changed from f161a34266 to ae24f87158, config revision is a48a7db0a2 [production]
20:36 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1293.eqiad.wmnet [production]
20:36 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1370.eqiad.wmnet [production]
20:36 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1378.eqiad.wmnet [production]
20:31 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1378.eqiad.wmnet [production]
20:31 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1370.eqiad.wmnet [production]
20:23 <mutante> mw1370, mw1378 - powercycling via DRAC [production]
20:21 <mutante> mw1370, mw1378 - again failing to reboot as the last step of reimaging script [production]
20:19 <jgleeson> updated civicrm from 1e9a86dd6e to f161a34266 [production]
20:13 <twentyafterfour@deploy1001> Synchronized php: group1 wikis to 1.36.0-wmf.30 (duration: 01m 02s) [production]
20:12 <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.36.0-wmf.30 [production]
20:05 <legoktm@cumin1001> conftool action : set/pooled=yes; selector: name=mw1324.eqiad.wmnet [production]
20:01 <ebernhardson@deploy1001> Finished deploy [wikimedia/discovery/analytics@1c5477d]: query_clicks: timestamp is now a reserved keyword (duration: 02m 19s) [production]
20:01 <legoktm@cumin1001> conftool action : set/pooled=yes; selector: name=mw1323.eqiad.wmnet [production]