401-450 of 10000 results (21ms)
2020-08-17 §
09:27 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
09:23 <jayme@cumin1001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) [production]
09:22 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
09:21 <jayme@cumin1001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) [production]
09:18 <_joe_> running a full apt-get upgrade on mw1379-1380 [production]
09:18 <_joe_> re-upgrading imagemagick on mw1378 [production]
09:16 <_joe_> upgrading packages on mw1377 [production]
09:14 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
09:06 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
09:06 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
09:05 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single [production]
08:25 <jayme> forcing a puppet run on all mw-api servers in eqiad - T260329 [production]
07:52 <_joe_> repooling mw1382 [production]
07:37 <_joe_> running the same test on mw1382 T260329 [production]
07:34 <_joe_> repooling mw1381 [production]
07:15 <_joe_> running the same test on mw1381 T260329 [production]
07:15 <_joe_> repooled mw1281 [production]
06:26 <_joe_> stop testing on mw1281, T260329 [production]
05:45 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
05:43 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime [production]
05:28 <marostegui> Stop mysql on db1099:3311, db1099:3318 for reimage [production]
05:28 <_joe_> depooling mw1281 for testing for T260329 [production]
05:25 <marostegui> Deploy schema change on db1139:3311 [production]
05:21 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1099:3311, db1099:3318 for reimage and MCR change', diff saved to https://phabricator.wikimedia.org/P12263 and previous config saved to /var/cache/conftool/dbconfig/20200817-052147-marostegui.json [production]
01:36 <wm-bot> <bd808> Testing a Freenode<->Discord bridge [tools.bridgebot]
2020-08-16 §
20:16 <andrewbogott> moving agent-qemu-1001 to a new host [integration]
19:50 <andrewbogott> moving integration-agent-docker-1002 to a new host [integration]
11:12 <gehel> repooling wdqs1004 - catched up on lag [production]
2020-08-15 §
21:18 <gehel> depooling wdqs1004 and restarting services, will wait to catch up on lag before repooling [production]
07:41 <legoktm> reloading zuul for https://gerrit.wikimedia.org/r/620426 [releng]
07:28 <legoktm> rebuilding rust image https://gerrit.wikimedia.org/r/620404 [releng]
06:19 <legoktm> reloading zuul for https://gerrit.wikimedia.org/r/620398 https://gerrit.wikimedia.org/r/620399 [releng]
05:47 <legoktm> building rust:1.45.2-2 image (https://gerrit.wikimedia.org/r/620396) [releng]
05:37 <legoktm> reloading zuul for https://gerrit.wikimedia.org/r/620395 [releng]
2020-08-14 §
19:47 <wm-bot> <lucaswerkmeister> deployed 13282d5404 (Bengali verb updates) [tools.lexeme-forms]
19:41 <effie> restart mwdebug1002 [production]
17:32 <Operator873> Operator873 restart CVNBot6-9 & 19 in @cvn-sw [cvn]
17:30 <Operator873> 12:30:23 @Operator873 CVNBot5 restart [cvn]
17:29 <Operator873> 12:27:01 @Operator873 CVNBot15 restart [cvn]
17:09 <bstorm> backing up the old proxy config to NFS and deleting paws-proxy-02 T211096 [paws]
17:04 <bstorm> deleting instances "tools-paws-master-01", "tools-paws-worker-1005", "tools-paws-worker-1006", "tools-paws-worker-1003", "tools-paws-worker-1002", "tools-paws-worker-1001", "tools-paws-worker-1007", "tools-paws-worker-1013", "tools-paws-worker-1016", "tools-paws-worker-1017", "tools-paws-worker-1010", "tools-paws-worker-1019" T211096 [tools.paws]
16:58 <cdanis> done deploying 'allow nameservers of Jio AS55836 to skip RPKI validation I9fcff8' to all routers T260449 [production]
16:44 <cdanis> ❌cdanis@cumin1001.eqiad.wmnet ~ 🕧☕ homer 'cr2-esams*' commit 'allow nameservers of Jio AS55836 to skip RPKI validation I9fcff8' [production]
16:39 <cdanis> ✔️ cdanis@cumin1001.eqiad.wmnet ~ 🕧☕ homer 'cr1-codfw*' commit 'allow nameservers of Jio AS55836 to skip RPKI validation I9fcff8' [production]
16:36 <cdanis> ❌cdanis@cumin1001.eqiad.wmnet ~ 🕧☕ homer 'cr2-codfw*' commit 'allow nameservers of Jio AS55836 to skip RPKI validation I9fcff8' [production]
13:06 <hashar> Successfully tagged docker-registry.discovery.wmnet/releng/rust:1.45.2-1 # T256827 [releng]
09:13 <fdans> restarting refine to apply T257860 [analytics]
02:41 <eileen> tools revision changed from 9a89f45974 to b4ebd1e564 [production]
00:49 <wm-bot> <bd808> Switched to node10 container and deployed https://github.com/toolforge/tool-hatjitsu/commit/d04e1d7 [tools.hatjitsu]
2020-08-13 §
23:39 <tzatziki> removing 3 files for legal compliance [production]