1001-1050 of 10000 results (35ms)
2018-09-11 §
13:21 <END> (PASS) - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki (exit_code=0) (volans@sarin) [production]
13:21 <START> - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki (volans@sarin) [production]
13:14 <END> (PASS) - Cookbook sre.switchdc.mediawiki.01-stop-maintenance (exit_code=0) (volans@sarin) [production]
13:14 <START> - Cookbook sre.switchdc.mediawiki.01-stop-maintenance (volans@sarin) [production]
13:12 <END> (PASS) - Cookbook sre.switchdc.mediawiki.00-reduce-ttl (exit_code=0) (volans@sarin) [production]
13:12 <START> - Cookbook sre.switchdc.mediawiki.00-reduce-ttl (volans@sarin) [production]
13:08 <volans> performing some additional switchdc live test [production]
13:02 <volans> upgraded spicerack to version 0.0.8 on sarin/neodymium - T199079 [production]
12:28 <gehel> restarting tilerator on maps1* (eqiad) - heap memory exceeded [production]
12:09 <moritzm> installing jq security updates on trusty [production]
12:01 <dereckson@deploy1001> Synchronized wmf-config/throttle.php: Update Informatika SZŠ Chomutov throttle rule (T203909) (duration: 00m 50s) [production]
12:00 <dereckson@deploy1001> sync-file aborted: Update Informatika SZŠ Chomutov throttle rule (duration: 00m 04s) [production]
11:49 <volans> uploaded spicerack_0.0.8-1{,+deb9u1} to apt.wikimedia.org {jessie,stretch}-wikimedia - T199079 [production]
11:37 <moritzm> restarting hhvm on mw1261-mw1265 to pick up curl security updates [production]
11:25 <zfilipin@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:455804|Set category collation to uca-et-u-kn on Estonian-language wikis (T202977)]] (duration: 00m 50s) [production]
10:37 <marostegui> Disable GTID on all codfw masters (sX, x1, esX) (not in db2040 as it is not enabled there) T189107 [production]
10:36 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1096:3315, db1100 (duration: 00m 49s) [production]
10:30 <tgr@deploy1001> Finished scap: T204018 update i18n on fixcopyrightwiki (duration: 31m 01s) [production]
10:27 <marostegui> db1096:3315 and db1100 were test pages - NO MORE TEST PAGES ARE EXPECTED FROM NOW ON - T200509 [production]
10:16 <marostegui> Stop replication on db2075 to test the paging (should not page) [production]
10:14 <marostegui> Stop replication on db1100 to test the paging [production]
10:03 <marostegui> Stop replication on db2084:3315 for alert testing [production]
09:59 <tgr@deploy1001> Started scap: T204018 update i18n on fixcopyrightwiki [production]
09:54 <marostegui> Stop replication on db1096:3315 for paging testing [production]
09:25 <moritzm> installing curl security updates [production]
08:39 <godog> repair xfs on sdh/sdc on ms-be2040 - T199198 [production]
08:27 <marostegui> Stop replication on db1100 for new alert testing (this should generate a page) T200509 [production]
08:25 <jynus> restarting replication on db2034 after testing dc switch replication sync step [production]
08:14 <marostegui> Stop replication on db1096:3315 for new alert testing (this should generate a page) T200509 [production]
08:13 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1096:3315, db1100 (duration: 00m 49s) [production]
08:13 <jynus> stopping replication on db2034 to test dc switch replication sync step [production]
08:12 <marostegui@deploy1001> sync-file aborted: Depool db1096:3315, db1100 (duration: 00m 08s) [production]
08:05 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2084:3315 and db2075 (duration: 00m 49s) [production]
07:49 <marostegui> Stop replication on db2075 for alert testing T200509 [production]
07:33 <marostegui> Stop replication on db2084:3315 for alert testing T200509 [production]
07:27 <marostegui> Disable puppet on all the DBs for alert testing - https://phabricator.wikimedia.org/T200509 [production]
05:57 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2084:3315 and db2075 (duration: 00m 51s) [production]
02:45 <l10nupdate@deploy1001> ResourceLoader cache refresh completed at Tue Sep 11 02:45:31 UTC 2018 (duration 10m 44s) [production]
02:44 <Krinkle> krinkle@mwmaint1001$ mwscript deleteEqualMessages.php --wiki fixcopyrightwiki --delete --lang-code='*' [production]
02:34 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.20) (duration: 13m 27s) [production]
01:00 <Krinkle> krinkle@mwmaint1001$ mwscript deleteEqualMessages.php --wiki fixcopyrightwiki --delete --lang-code='*' [production]
2018-09-10 §
22:56 <andrewbogott> rebooting/reimaging labvirt1019 and 1020 for T204003 [production]
22:43 <ejegg> updated CiviCRM from 52873b5dcc to 2a3583cca1 [production]
22:00 <ejegg> updated payments-wiki from 05d796e844 to 44b9409104 [production]
21:24 <bawolff@deploy1001> Synchronized php-1.32.0-wmf.20/extensions/OpenStackManager/special/SpecialNovaSudoer.php: T203885 (duration: 00m 50s) [production]
20:29 <volans> running warmup script against codfw appservers [production]
19:03 <brion> stopped requeueTranscodes.php job on mwmaint1001.eqiad pending dc switch [production]
17:42 <XioNoX> trunk cloud-instances1-b-eqiad to cloudnet1003/4:eth1 - T202636 [production]
17:03 <elukey> stop ircecho to avoid excessive spamming [production]
16:37 <elukey> rolling restart of aqs on aqs* to pick new druid backend settings [production]