|
2022-08-04
ยง
|
| 18:35 |
<dzahn@cumin2002> |
END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for mw2366.codfw.wmnet |
[production] |
| 18:35 |
<dzahn@cumin2002> |
START - Cookbook sre.hosts.remove-downtime for mw2366.codfw.wmnet |
[production] |
| 18:34 |
<dzahn@cumin2002> |
conftool action : set/pooled=yes; selector: dc=codfw,name=mw2366.codfw.wmnet |
[production] |
| 18:31 |
<ottomata> |
dropping medawiki_web_ui_interactions hive tables and data - T314151 |
[analytics] |
| 18:30 |
<dzahn@cumin2002> |
conftool action : set/pooled=yes; selector: dc=codfw,name=mw2279.codfw.wmnet |
[production] |
| 18:30 |
<dzahn@cumin2002> |
conftool action : set/pooled=yes; selector: dc=codfw,name=mw2278.codfw.wmnet |
[production] |
| 18:29 |
<dzahn@cumin2002> |
conftool action : set/pooled=yes; selector: dc=codfw,name=mw2277.codfw.wmnet |
[production] |
| 18:29 |
<dzahn@cumin2002> |
END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for mw2276.codfw.wmnet |
[production] |
| 18:29 |
<dzahn@cumin2002> |
START - Cookbook sre.hosts.remove-downtime for mw2276.codfw.wmnet |
[production] |
| 18:29 |
<dzahn@cumin2002> |
END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for mw2275.codfw.wmnet |
[production] |
| 18:29 |
<dzahn@cumin2002> |
START - Cookbook sre.hosts.remove-downtime for mw2275.codfw.wmnet |
[production] |
| 18:29 |
<dzahn@cumin2002> |
END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for mw2274.codfw.wmnet |
[production] |
| 18:29 |
<dzahn@cumin2002> |
START - Cookbook sre.hosts.remove-downtime for mw2274.codfw.wmnet |
[production] |
| 18:29 |
<dzahn@cumin2002> |
END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for mw2273.codfw.wmnet |
[production] |
| 18:29 |
<dzahn@cumin2002> |
START - Cookbook sre.hosts.remove-downtime for mw2273.codfw.wmnet |
[production] |
| 18:26 |
<milimetric@deploy1002> |
Finished deploy [analytics/refinery@2553288]: Regular analytics weekly train [analytics/refinery@2553288] (duration: 02m 39s) |
[production] |
| 18:24 |
<dzahn@cumin2002> |
END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for mw2272.codfw.wmnet |
[production] |
| 18:24 |
<dzahn@cumin2002> |
START - Cookbook sre.hosts.remove-downtime for mw2272.codfw.wmnet |
[production] |
| 18:24 |
<dzahn@cumin2002> |
END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for mw2271.codfw.wmnet |
[production] |
| 18:24 |
<dzahn@cumin2002> |
START - Cookbook sre.hosts.remove-downtime for mw2271.codfw.wmnet |
[production] |
| 18:23 |
<milimetric@deploy1002> |
Started deploy [analytics/refinery@2553288]: Regular analytics weekly train [analytics/refinery@2553288] |
[production] |
| 18:23 |
<milimetric@deploy1002> |
Finished deploy [analytics/refinery@2553288]: Regular analytics weekly train [analytics/refinery@2553288] (duration: 01m 32s) |
[production] |
| 18:23 |
<dzahn@cumin2002> |
conftool action : set/pooled=yes; selector: dc=codfw,name=mw2276.codfw.wmnet |
[production] |
| 18:23 |
<dzahn@cumin2002> |
conftool action : set/pooled=yes; selector: dc=codfw,name=mw2275.codfw.wmnet |
[production] |
| 18:23 |
<dzahn@cumin2002> |
conftool action : set/pooled=yes; selector: dc=codfw,name=mw2274.codfw.wmnet |
[production] |
| 18:22 |
<dzahn@cumin2002> |
conftool action : set/pooled=yes; selector: dc=codfw,name=mw2273.codfw.wmnet |
[production] |
| 18:22 |
<dzahn@cumin2002> |
conftool action : set/pooled=yes; selector: dc=codfw,name=mw2272.codfw.wmnet |
[production] |
| 18:22 |
<Emperor> |
shutdown moss-fe2001.codfw.wmnet,ms-fe2011.codfw.wmnet,ms-be20[34,35,42,48,68].codfw.wmnet PDU work T310145 |
[production] |
| 18:22 |
<mvernon@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on 8 hosts with reason: PDU work |
[production] |
| 18:21 |
<milimetric@deploy1002> |
Started deploy [analytics/refinery@2553288]: Regular analytics weekly train [analytics/refinery@2553288] |
[production] |
| 18:21 |
<mvernon@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on 8 hosts with reason: PDU work |
[production] |
| 18:21 |
<milimetric@deploy1002> |
Finished deploy [analytics/refinery@2553288]: Regular analytics weekly train [analytics/refinery@2553288] (duration: 00m 03s) |
[production] |
| 18:21 |
<milimetric@deploy1002> |
Started deploy [analytics/refinery@2553288]: Regular analytics weekly train [analytics/refinery@2553288] |
[production] |
| 18:21 |
<milimetric@deploy1002> |
Finished deploy [analytics/refinery@2553288]: Regular analytics weekly train [analytics/refinery@2553288] (duration: 00m 03s) |
[production] |
| 18:21 |
<milimetric@deploy1002> |
Started deploy [analytics/refinery@2553288]: Regular analytics weekly train [analytics/refinery@2553288] |
[production] |
| 18:20 |
<milimetric@deploy1002> |
Finished deploy [analytics/refinery@2553288]: Regular analytics weekly train [analytics/refinery@2553288] (duration: 01m 49s) |
[production] |
| 18:20 |
<mvernon@cumin1001> |
END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for 9 hosts |
[production] |
| 18:20 |
<mvernon@cumin1001> |
START - Cookbook sre.hosts.remove-downtime for 9 hosts |
[production] |
| 18:19 |
<milimetric> |
scap deploying refinery host by host after Ben cleaned up the repos with "git checkout master" |
[analytics] |
| 18:19 |
<milimetric@deploy1002> |
Started deploy [analytics/refinery@2553288]: Regular analytics weekly train [analytics/refinery@2553288] |
[production] |
| 18:14 |
<mutante> |
mw2272 and upwards: scap pull, checking monitoring, repooling.. one by one |
[production] |
| 18:13 |
<dzahn@cumin2002> |
conftool action : set/pooled=yes; selector: dc=codfw,name=mw2271.codfw.wmnet |
[production] |
| 18:12 |
<btullis@deploy1002> |
Finished deploy [analytics/refinery@2553288]: Regular analytics weekly train [analytics/refinery@2553288] (duration: 00m 51s) |
[production] |
| 18:11 |
<btullis> |
btullis@deploy1002:/srv/deployment/analytics/refinery$ scap deploy -l stat1008.eqiad.wmnet "Regular analytics weekly train [analytics/refinery@$(git rev-parse --short HEAD)]" |
[analytics] |
| 18:11 |
<btullis@deploy1002> |
Started deploy [analytics/refinery@2553288]: Regular analytics weekly train [analytics/refinery@2553288] |
[production] |
| 18:06 |
<btullis@deploy1002> |
Finished deploy [analytics/refinery@2553288]: Regular analytics weekly train [analytics/refinery@2553288] (duration: 01m 54s) |
[production] |
| 18:05 |
<btullis> |
we are re-deploying refinery to an-launcher1002 with the command above |
[analytics] |
| 18:04 |
<btullis> |
btullis@deploy1002:/srv/deployment/analytics/refinery$ scap deploy -l an-launcher1002.eqiad.wmnet "Regular analytics weekly train [analytics/refinery@$(git rev-parse --short HEAD)]" |
[analytics] |
| 18:04 |
<btullis@deploy1002> |
Started deploy [analytics/refinery@2553288]: Regular analytics weekly train [analytics/refinery@2553288] |
[production] |
| 18:02 |
<btullis> |
analytics-deploy@an-launcher1002:/srv/deployment/analytics/refinery$ git checkout master |
[analytics] |