801-850 of 10000 results (26ms)
2021-01-12 ยง
17:20 <herron> roll restarting eqiad/codfw low-traffic pybals for kibana-next -> kibana7 rename [production]
17:11 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
17:09 <jynus> shutting down db2132, db2078:m1 for m1 codfw replica reprovisioning T270877 [production]
17:09 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
17:09 <moritzm> rebooting people1002 (people.wikimedia.org) [production]
16:56 <moritzm> reinstalling bast3005 with correct DHCP settings [production]
16:39 <herron@puppetmaster1001> conftool action : set/pooled=yes:weight=10; selector: cluster=kibana7,service=kibana7 [production]
16:37 <ema> cp5008: ats-backend-restart to apply jit.off(true, true) to all lua scripts T265625 [production]
16:33 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
16:31 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
16:18 <herron@puppetmaster1001> conftool action : set/weight=10; selector: name=logstash2031.codfw.wmnet [production]
15:56 <ema> cp5008: ats-backend-restart to apply jit.off(true, true) in default.lua T265625 [production]
15:55 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
15:53 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
15:52 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on ms-be2055.codfw.wmnet with reason: reboot [production]
15:52 <filippo@cumin1001> START - Cookbook sre.hosts.downtime for 4:00:00 on ms-be2055.codfw.wmnet with reason: reboot [production]
15:22 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on ms-be2031.codfw.wmnet with reason: test unattended reboot [production]
15:22 <filippo@cumin1001> START - Cookbook sre.hosts.downtime for 4:00:00 on ms-be2031.codfw.wmnet with reason: test unattended reboot [production]
14:19 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
14:17 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
14:14 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
14:05 <liw@deploy1001> rebuilt and synchronized wikiversions files: group0 wikis to 1.36.0-wmf.26 [production]
14:00 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
13:55 <moritzm> draining ganeti3003 for eventual reboot [production]
13:53 <moritzm> failover ganeti master in esams to ganeti3002 [production]
13:44 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
13:39 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
13:33 <moritzm> draining ganeti3002 for eventual reboot [production]
12:19 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
12:12 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
12:08 <moritzm> draining ganeti3001 for eventual reboot [production]
11:22 <moritzm> installing edk2 security updates [production]
10:51 <godog> swift eqiad-prod: add weight to ms-be106[0-3] - T268435 [production]
10:28 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on cloudnet1004.eqiad.wmnet with reason: T271058 [production]
10:28 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on cloudnet1004.eqiad.wmnet with reason: T271058 [production]
10:26 <moritzm> installing systemd bugfix update from Buster 10.7 point release [production]
10:15 <liw@deploy1001> Finished scap: testwikis wikis to 1.36.0-wmf.26 (duration: 67m 18s) [production]
10:13 <marostegui> Restart mysql on db1138 to pick up new config T271427 T271106 [production]
10:12 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1138', diff saved to https://phabricator.wikimedia.org/P13736 and previous config saved to /var/cache/conftool/dbconfig/20210112-101211-marostegui.json [production]
09:28 <liw@deploy1001> Started scap: testwikis wikis to 1.36.0-wmf.26 [production]
09:05 <marostegui@cumin1001> dbctl commit (dc=all): 'db1112 (re)pooling @ 100%: After schema change', diff saved to https://phabricator.wikimedia.org/P13732 and previous config saved to /var/cache/conftool/dbconfig/20210112-090533-root.json [production]
08:50 <marostegui@cumin1001> dbctl commit (dc=all): 'db1112 (re)pooling @ 75%: After schema change', diff saved to https://phabricator.wikimedia.org/P13731 and previous config saved to /var/cache/conftool/dbconfig/20210112-085030-root.json [production]
08:49 <reedy@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: T271755 (duration: 00m 57s) [production]
08:47 <liw> 1.36.0-wmf.26 was branched at e6ad9ab7713ee33c30cd7c17762737870dc8fd08 for T267419 [production]
08:40 <marostegui> Sanitize bclwiktionary diqwiktionary niawiki niawiktionary diqwiktionary on db1124 db2094 db11154 T270280 T270276 T270414 T270410 T271261 [production]
08:35 <marostegui@cumin1001> dbctl commit (dc=all): 'db1112 (re)pooling @ 50%: After schema change', diff saved to https://phabricator.wikimedia.org/P13730 and previous config saved to /var/cache/conftool/dbconfig/20210112-083526-root.json [production]
08:30 <moritzm> installing remaining curl security updates on stretch [production]
08:21 <marostegui> Deploy schema change on s3 eqiad master - T270187 [production]
08:20 <marostegui@cumin1001> dbctl commit (dc=all): 'db1112 (re)pooling @ 25%: After schema change', diff saved to https://phabricator.wikimedia.org/P13729 and previous config saved to /var/cache/conftool/dbconfig/20210112-082023-root.json [production]
08:04 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1112', diff saved to https://phabricator.wikimedia.org/P13728 and previous config saved to /var/cache/conftool/dbconfig/20210112-080419-marostegui.json [production]