701-750 of 10000 results (20ms)
2021-03-30 ยง
20:28 <legoktm@deploy1002> conftool action : set/pooled=yes; selector: cluster=videoscaler,name=mw1303.eqiad.wmnet [production]
20:28 <legoktm@deploy1002> conftool action : set/pooled=no; selector: cluster=jobrunner,name=mw1307.eqiad.wmnet [production]
20:28 <legoktm@deploy1002> conftool action : set/pooled=no; selector: cluster=jobrunner,name=mw1306.eqiad.wmnet [production]
20:27 <legoktm@deploy1002> conftool action : set/pooled=no; selector: cluster=jobrunner,name=mw1305.eqiad.wmnet [production]
20:27 <legoktm@deploy1002> conftool action : set/pooled=no; selector: cluster=jobrunner,name=mw1304.eqiad.wmnet [production]
20:27 <legoktm@deploy1002> conftool action : set/pooled=no; selector: cluster=jobrunner,name=mw1303.eqiad.wmnet [production]
20:26 <twentyafterfour> preparing to deploy phatality upgrade to kibana cluster [production]
20:25 <legoktm@deploy1002> conftool action : set/pooled=yes; selector: cluster=jobrunner,name=mw1296.eqiad.wmnet [production]
20:25 <legoktm@deploy1002> conftool action : set/pooled=yes; selector: cluster=jobrunner,name=mw1298.eqiad.wmnet [production]
20:25 <legoktm@deploy1002> conftool action : set/pooled=yes; selector: cluster=jobrunner,name=mw1299.eqiad.wmnet [production]
20:21 <joal@deploy1002> Finished deploy [analytics/refinery@1a53e9a] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@1a53e9a] (duration: 04m 29s) [production]
20:20 <legoktm@deploy1002> conftool action : set/pooled=no; selector: cluster=videoscaler,name=mw1299.eqiad.wmnet [production]
20:20 <legoktm@deploy1002> conftool action : set/pooled=no; selector: cluster=videoscaler,name=mw1298.eqiad.wmnet [production]
20:20 <legoktm@deploy1002> conftool action : set/pooled=no; selector: cluster=videoscaler,name=mw1296.eqiad.wmnet [production]
20:16 <joal@deploy1002> Started deploy [analytics/refinery@1a53e9a] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@1a53e9a] [production]
20:16 <joal@deploy1002> Finished deploy [analytics/refinery@1a53e9a] (thin): Regular analytics weekly train THIN [analytics/refinery@1a53e9a] (duration: 00m 07s) [production]
20:16 <joal@deploy1002> Started deploy [analytics/refinery@1a53e9a] (thin): Regular analytics weekly train THIN [analytics/refinery@1a53e9a] [production]
20:15 <joal@deploy1002> Finished deploy [analytics/refinery@1a53e9a]: Regular analytics weekly train [analytics/refinery@1a53e9a] (duration: 17m 11s) [production]
20:02 <twentyafterfour> when syncing 1.36.0-wmf.37 promote to testwikis, one server failed: server mw1298.eqiad.wmnet and two more appear to be hung because scap is stuck at 2 left 99% without making any progress for a long time now. refs T278343 [production]
19:58 <bblack@cumin1001> conftool action : set/pooled=yes; selector: name=cp1087.eqiad.wmnet [production]
19:58 <joal@deploy1002> Started deploy [analytics/refinery@1a53e9a]: Regular analytics weekly train [analytics/refinery@1a53e9a] [production]
19:58 <bblack> repool cp1087 - T278729 [production]
19:13 <twentyafterfour@deploy1002> Started scap: testwikis wikis to 1.36.0-wmf.37 refs T278343 [production]
18:15 <pt1979@cumin2001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
18:09 <pt1979@cumin2001> START - Cookbook sre.dns.netbox [production]
17:22 <legoktm> moved mw[1293-1295] to jobrunners and mw[1300-1302] to videoscalers [production]
17:22 <legoktm@deploy1002> conftool action : set/pooled=yes; selector: cluster=videoscaler,name=mw1302.eqiad.wmnet [production]
17:22 <legoktm@deploy1002> conftool action : set/pooled=yes; selector: cluster=videoscaler,name=mw1301.eqiad.wmnet [production]
17:21 <legoktm@deploy1002> conftool action : set/pooled=yes; selector: cluster=videoscaler,name=mw1300.eqiad.wmnet [production]
17:21 <legoktm@deploy1002> conftool action : set/pooled=no; selector: cluster=jobrunner,name=mw1302.eqiad.wmnet [production]
17:21 <legoktm@deploy1002> conftool action : set/pooled=no; selector: cluster=jobrunner,name=mw1301.eqiad.wmnet [production]
17:21 <legoktm@deploy1002> conftool action : set/pooled=no; selector: cluster=jobrunner,name=mw1300.eqiad.wmnet [production]
17:21 <legoktm@deploy1002> conftool action : set/pooled=yes; selector: cluster=jobrunner,name=mw1295.eqiad.wmnet [production]
17:21 <legoktm@deploy1002> conftool action : set/pooled=yes; selector: cluster=jobrunner,name=mw1294.eqiad.wmnet [production]
17:21 <legoktm@deploy1002> conftool action : set/pooled=yes; selector: cluster=jobrunner,name=mw1293.eqiad.wmnet [production]
17:19 <legoktm> killed all ffmpeg on mw1294 [production]
17:17 <legoktm@deploy1002> conftool action : set/pooled=no; selector: cluster=videoscaler,name=mw1295.eqiad.wmnet [production]
17:17 <legoktm@deploy1002> conftool action : set/pooled=no; selector: cluster=videoscaler,name=mw1293.eqiad.wmnet [production]
17:17 <legoktm@deploy1002> conftool action : set/pooled=no; selector: cluster=videoscaler,name=mw1294.eqiad.wmnet [production]
17:13 <mbsantos@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mobileapps' for release 'production' . [production]
17:12 <mbsantos@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'production' . [production]
17:10 <mbsantos@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'mobileapps' for release 'staging' . [production]
17:08 <mbsantos@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'proton' for release 'production' . [production]
17:05 <mbsantos@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'proton' for release 'production' . [production]
17:02 <mbsantos@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'proton' for release 'production' . [production]
16:40 <effie> enable puppet on mw* hosts [production]
16:10 <mutante> mw1296 - started ferm [production]
16:10 <mutante> mw1308 - started ferm [production]
16:07 <akosiaris> split jobrunners/videoscalers clusters in conftool. mw12* become videoscalers, mw13* become jobrunners, killing ffmpeg on mw13* [production]
16:07 <mutante> mw1309 - systemctl start ferm [production]