|
2021-03-30
ยง
|
| 19:58 |
<joal@deploy1002> |
Started deploy [analytics/refinery@1a53e9a]: Regular analytics weekly train [analytics/refinery@1a53e9a] |
[production] |
| 19:58 |
<bblack> |
repool cp1087 - T278729 |
[production] |
| 19:57 |
<joal> |
Deploying refinery using scap |
[analytics] |
| 19:57 |
<joal> |
Refinery-source released to archiva and new jars commited to refinery (v0.1.3) |
[analytics] |
| 19:13 |
<twentyafterfour@deploy1002> |
Started scap: testwikis wikis to 1.36.0-wmf.37 refs T278343 |
[production] |
| 18:15 |
<pt1979@cumin2001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 18:09 |
<pt1979@cumin2001> |
START - Cookbook sre.dns.netbox |
[production] |
| 18:05 |
<Majavah> |
remove {trysty,precise}-deployment-prep repos from deployment-deploy01 aptly |
[releng] |
| 17:51 |
<Majavah> |
arm deployment-deploy01 keyholder with all the keys |
[releng] |
| 17:22 |
<legoktm> |
moved mw[1293-1295] to jobrunners and mw[1300-1302] to videoscalers |
[production] |
| 17:22 |
<legoktm@deploy1002> |
conftool action : set/pooled=yes; selector: cluster=videoscaler,name=mw1302.eqiad.wmnet |
[production] |
| 17:22 |
<legoktm@deploy1002> |
conftool action : set/pooled=yes; selector: cluster=videoscaler,name=mw1301.eqiad.wmnet |
[production] |
| 17:21 |
<legoktm@deploy1002> |
conftool action : set/pooled=yes; selector: cluster=videoscaler,name=mw1300.eqiad.wmnet |
[production] |
| 17:21 |
<legoktm@deploy1002> |
conftool action : set/pooled=no; selector: cluster=jobrunner,name=mw1302.eqiad.wmnet |
[production] |
| 17:21 |
<legoktm@deploy1002> |
conftool action : set/pooled=no; selector: cluster=jobrunner,name=mw1301.eqiad.wmnet |
[production] |
| 17:21 |
<legoktm@deploy1002> |
conftool action : set/pooled=no; selector: cluster=jobrunner,name=mw1300.eqiad.wmnet |
[production] |
| 17:21 |
<legoktm@deploy1002> |
conftool action : set/pooled=yes; selector: cluster=jobrunner,name=mw1295.eqiad.wmnet |
[production] |
| 17:21 |
<legoktm@deploy1002> |
conftool action : set/pooled=yes; selector: cluster=jobrunner,name=mw1294.eqiad.wmnet |
[production] |
| 17:21 |
<legoktm@deploy1002> |
conftool action : set/pooled=yes; selector: cluster=jobrunner,name=mw1293.eqiad.wmnet |
[production] |
| 17:19 |
<legoktm> |
killed all ffmpeg on mw1294 |
[production] |
| 17:17 |
<legoktm@deploy1002> |
conftool action : set/pooled=no; selector: cluster=videoscaler,name=mw1295.eqiad.wmnet |
[production] |
| 17:17 |
<legoktm@deploy1002> |
conftool action : set/pooled=no; selector: cluster=videoscaler,name=mw1293.eqiad.wmnet |
[production] |
| 17:17 |
<legoktm@deploy1002> |
conftool action : set/pooled=no; selector: cluster=videoscaler,name=mw1294.eqiad.wmnet |
[production] |
| 17:13 |
<mbsantos@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mobileapps' for release 'production' . |
[production] |
| 17:12 |
<mbsantos@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'production' . |
[production] |
| 17:10 |
<mbsantos@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'mobileapps' for release 'staging' . |
[production] |
| 17:08 |
<mbsantos@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'proton' for release 'production' . |
[production] |
| 17:07 |
<razzi> |
rebalance kafka partitions for webrequest_text partitions 5 and 6 |
[analytics] |
| 17:05 |
<mbsantos@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'proton' for release 'production' . |
[production] |
| 17:02 |
<mbsantos@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'proton' for release 'production' . |
[production] |
| 16:40 |
<effie> |
enable puppet on mw* hosts |
[production] |
| 16:15 |
<bstorm> |
added `labstore::traffic_shaping::egress: 800mbps` to tools-static prefix T278539 |
[tools] |
| 16:10 |
<mutante> |
mw1296 - started ferm |
[production] |
| 16:10 |
<mutante> |
mw1308 - started ferm |
[production] |
| 16:07 |
<akosiaris> |
split jobrunners/videoscalers clusters in conftool. mw12* become videoscalers, mw13* become jobrunners, killing ffmpeg on mw13* |
[production] |
| 16:07 |
<mutante> |
mw1309 - systemctl start ferm |
[production] |
| 16:07 |
<akosiaris@cumin1001> |
conftool action : set/pooled=no; selector: dc=eqiad,cluster=jobrunner,name=mw12.* |
[production] |
| 16:06 |
<akosiaris@cumin1001> |
conftool action : set/pooled=no; selector: dc=eqiad,cluster=videoscaler,name=mw13.* |
[production] |
| 16:06 |
<akosiaris@cumin1001> |
conftool action : set/pooled=yes; selector: dc=eqiad,cluster=videoscaler,name=mw12.* |
[production] |
| 15:59 |
<akosiaris> |
depool a number of hosts from videoscalers |
[production] |
| 15:59 |
<akosiaris@cumin1001> |
conftool action : set/pooled=no; selector: dc=eqiad,cluster=videoscaler,name=mw12.* |
[production] |
| 15:55 |
<legoktm@deploy1002> |
conftool action : set/pooled=no; selector: name=mw1308.eqiad.wmnet,service=jobrunner |
[production] |
| 15:55 |
<legoktm@deploy1002> |
conftool action : set/pooled=no; selector: name=mw1307.eqiad.wmnet,service=jobrunner |
[production] |
| 15:44 |
<arturo> |
shutoff tools-static-12/13 (T278539) |
[tools] |
| 15:42 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=aqs1004.eqiad.wmnet |
[production] |
| 15:41 |
<arturo> |
point horizon web proxy `tools-static.wmflabs.org` to tools-static-14 (T278539) |
[tools] |
| 15:37 |
<arturo> |
add `mount_nfs: true` to tools-static prefix (T2778539) |
[tools] |
| 15:29 |
<hnowlan> |
moving all test tables out of cassandra directories on aqs hosts |
[production] |
| 15:26 |
<arturo> |
create VM tools-static-14 with Debian Buster image (T278539) |
[tools] |
| 14:59 |
<effie> |
disable puppet on mediawiki servers to deploy 663565 |
[production] |