3251-3300 of 10000 results (33ms)
2020-10-01 §
05:45 <marostegui> Stop MySQL on es2011 T264261 [production]
05:43 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool es2011 T264261', diff saved to https://phabricator.wikimedia.org/P12866 and previous config saved to /var/cache/conftool/dbconfig/20201001-054335-marostegui.json [production]
05:38 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
05:35 <marostegui@cumin1001> START - Cookbook sre.hosts.decommission [production]
05:29 <marostegui> Deploy schema change on s3 (testwikidatawiki) T264109 [production]
05:19 <marostegui> Repool labsdb1011 [production]
04:20 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
04:18 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
01:27 <krinkle@deploy1001> Synchronized php-1.36.0-wmf.10/includes/parser/: Ia3357b2f593c (duration: 00m 58s) [production]
01:12 <krinkle@deploy1001> Synchronized wmf-config/CommonSettings.php: 1721d2aa0 - Reject ParserCache entries from the last wmf.11 deployment (duration: 05m 13s) [production]
2020-09-30 §
22:52 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
22:50 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
22:12 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
22:10 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
21:46 <cdanis> depool mw2356 and mw2319 [production]
21:45 <eileen> civicrm revision changed from 5a53bfe6ed to 256adda03c, config revision is 646817a2c0 [production]
21:23 <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: rollback group0 also [production]
21:19 <ejegg> updated fundraising CiviCRM from 6e843649ac to 5a53bfe6ed [production]
21:04 <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: rollback [production]
21:00 <twentyafterfour@deploy1001> scap failed: average error rate on 5/6 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/e474f13ffac6b8c3bf919c4aeafc8c9b for details) [production]
20:58 <twentyafterfour@deploy1001> Synchronized php: group1 wikis to 1.36.0-wmf.11 (duration: 01m 20s) [production]
20:56 <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.36.0-wmf.11 [production]
20:47 <mutante> temp disabling puppet on C:profile::swift::stats_reporter hosts, applying gerrit:631158 refactoring change [production]
20:36 <mutante> temp disabling puppet on swift::storage (swift-be) hosts, applying gerrit:631157 refactoring change [production]
19:21 <mutante> activating DHCP and squid on install[345]001.wikimedia.org [production]
19:12 <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: group0 wikis to 1.36.0-wmf.11 [production]
19:01 <effie> disable puppet on mw2271 and use onhost memcached - T263958 [production]
19:00 <hoo@deploy1001> Synchronized wmf-config/: Revert "labs: Turn on termbox v2 on wikidatawiki" (T264066) (duration: 00m 58s) [production]
18:58 <hoo@deploy1001> Synchronized wmf-config/Wikibase.php: Revert "labs: Turn on termbox v2 on wikidatawiki" (T264066) (duration: 00m 58s) [production]
18:38 <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable and configure GrowthExperiments on svwiki (T257220) (duration: 00m 58s) [production]
18:36 <bblack> lvs1016 pybal diff alerts downtimed in icinga for ~48h to reduce annoying flappy alert spam, with reference to https://phabricator.wikimedia.org/T264227 [production]
18:31 <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable GrowthExperiments for newcomers on ptwiki (T225027) (duration: 00m 58s) [production]
18:28 <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Put search in header for anons on all wikis, not just desktop-improvements wikis (T263032) (duration: 00m 59s) [production]
18:14 <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable clientError on Wikidata and all Wikipedias except enwiki (T255585) (duration: 00m 58s) [production]
18:08 <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Move search in header for anons (T263032) (duration: 00m 59s) [production]
17:52 <bblack> lvs1016: restart pybal [production]
17:04 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
17:02 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
17:01 <hnowlan> finished adding restbase2018-a to the cassandra cluster [production]
16:37 <pt1979@cumin2001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:33 <cicalese@deploy1001> Synchronized wmf-config/CommonSettings-labs.php: Add beta config for API Portal/OAuth communications (duration: 00m 58s) [production]
16:31 <pt1979@cumin2001> START - Cookbook sre.dns.netbox [production]
16:21 <mutante> re-enabled puppet on install2003 [production]
16:21 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:20 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:20 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:20 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:28 <moritzm> removed librsvg 2.40.20-3+wmf1+stretch1 from component/thumbor, superseded by 2.40.21-0+deb9u1 released via stretch-security [production]
14:23 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:22 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime [production]