451-500 of 10000 results (21ms)
2020-08-31 §
07:30 <moritzm> installing squid security updates [production]
07:24 <moritzm> installing openexr security updates on buster [production]
07:12 <marostegui> Sanitize jawikivoyage on db2094:3325 and db1124:3325 T260482 [production]
06:30 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
06:27 <elukey@cumin1001> START - Cookbook sre.hosts.downtime [production]
06:06 <elukey> reimage kafka-jumbo1005 to Debian Buster [production]
05:21 <marostegui> Reload haproxy on dbproxy1017 and dbproxy1021 to test db1128 [production]
2020-08-30 §
16:13 <herron> restarted eqiad v5 logstashes [production]
2020-08-29 §
18:05 <Amir1> end of ladsgroup@mwmaint1002:~$ foreachwikiindblist wikidataclient extensions/Wikibase/lib/maintenance/populateSitesTable.php --force-protocol https (T261451) [production]
17:45 <Amir1> start of ladsgroup@mwmaint1002:~$ foreachwikiindblist wikidataclient extensions/Wikibase/lib/maintenance/populateSitesTable.php --force-protocol https (T261451) [production]
2020-08-28 §
21:53 <ryankemper> `sudo systemctl reload nginx.service` on `cloudelastic100[5,6].wikimedia.org` to try to resolve certificate warning issues [production]
19:11 <andrewbogott> rebooting cloudvirt1006. It's a spare, unused system but showing a bus error and icinga alerts; not worth saving if it needs saving [production]
17:39 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
17:39 <mutante> shutting down mw2196 [production]
17:37 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
16:40 <rzl> switchdc live test complete [production]
16:36 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.08-update-tendril (exit_code=0) [production]
16:35 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.08-update-tendril [production]
16:35 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.08-start-maintenance (exit_code=0) [production]
16:34 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.08-start-maintenance [production]
16:33 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.08-restore-ttl (exit_code=0) [production]
16:33 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.08-restore-ttl [production]
16:33 <rzl@cumin1001> END (FAIL) - Cookbook sre.switchdc.mediawiki.08-restore-ttl (exit_code=99) [production]
16:33 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.08-restore-ttl [production]
16:29 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.07-set-readwrite (exit_code=0) [production]
16:29 <rzl@cumin1001> [DRY-RUN] MediaWiki read-only period ends at: 2020-08-28 16:29:24.432463 [production]
16:29 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.07-set-readwrite [production]
16:29 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite (exit_code=0) [production]
16:29 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite [production]
16:29 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.05-invert-redis-sessions (exit_code=0) [production]
16:29 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.05-invert-redis-sessions [production]
16:29 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki (exit_code=0) [production]
16:29 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki [production]
16:28 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.03-set-db-readonly (exit_code=0) [production]
16:28 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.03-set-db-readonly [production]
16:28 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.02-set-readonly (exit_code=0) [production]
16:28 <rzl@cumin1001> [DRY-RUN] MediaWiki read-only period starts at: 2020-08-28 16:28:07.882663 [production]
16:28 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.02-set-readonly [production]
16:19 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.01-stop-maintenance (exit_code=0) [production]
16:19 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.01-stop-maintenance [production]
16:13 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.00-warmup-caches (exit_code=0) [production]
16:12 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.00-warmup-caches [production]
16:09 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.00-reduce-ttl (exit_code=0) [production]
16:09 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.00-reduce-ttl [production]
16:08 <rzl@cumin1001> END (FAIL) - Cookbook sre.switchdc.mediawiki.00-reduce-ttl (exit_code=99) [production]
16:08 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.00-reduce-ttl [production]
16:07 <rzl@cumin1001> END (PASS) - Cookbook sre.switchdc.mediawiki.00-disable-puppet (exit_code=0) [production]
16:07 <rzl@cumin1001> START - Cookbook sre.switchdc.mediawiki.00-disable-puppet [production]
16:06 <rzl> starting one more live test of the data center switchover automation, no production impact is expected but there will be some SAL noise [production]
14:22 <moritzm> installing Java security updates on kafka/main and Logstash(5) clusters [production]