651-700 of 10000 results (47ms)
2018-09-11 §
08:05 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2084:3315 and db2075 (duration: 00m 49s) [production]
07:49 <marostegui> Stop replication on db2075 for alert testing T200509 [production]
07:33 <marostegui> Stop replication on db2084:3315 for alert testing T200509 [production]
07:27 <marostegui> Disable puppet on all the DBs for alert testing - https://phabricator.wikimedia.org/T200509 [production]
05:57 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2084:3315 and db2075 (duration: 00m 51s) [production]
02:45 <l10nupdate@deploy1001> ResourceLoader cache refresh completed at Tue Sep 11 02:45:31 UTC 2018 (duration 10m 44s) [production]
02:44 <Krinkle> krinkle@mwmaint1001$ mwscript deleteEqualMessages.php --wiki fixcopyrightwiki --delete --lang-code='*' [production]
02:34 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.20) (duration: 13m 27s) [production]
01:00 <Krinkle> krinkle@mwmaint1001$ mwscript deleteEqualMessages.php --wiki fixcopyrightwiki --delete --lang-code='*' [production]
2018-09-10 §
22:56 <andrewbogott> rebooting/reimaging labvirt1019 and 1020 for T204003 [production]
22:43 <ejegg> updated CiviCRM from 52873b5dcc to 2a3583cca1 [production]
22:00 <ejegg> updated payments-wiki from 05d796e844 to 44b9409104 [production]
21:24 <bawolff@deploy1001> Synchronized php-1.32.0-wmf.20/extensions/OpenStackManager/special/SpecialNovaSudoer.php: T203885 (duration: 00m 50s) [production]
20:29 <volans> running warmup script against codfw appservers [production]
19:03 <brion> stopped requeueTranscodes.php job on mwmaint1001.eqiad pending dc switch [production]
17:42 <XioNoX> trunk cloud-instances1-b-eqiad to cloudnet1003/4:eth1 - T202636 [production]
17:03 <elukey> stop ircecho to avoid excessive spamming [production]
16:37 <elukey> rolling restart of aqs on aqs* to pick new druid backend settings [production]
15:24 <gehel> reducing elasticsearch low watermark to 75% on cirrus / eqiad cluster [production]
15:07 <elukey> reboot analytics100[1,2] for kernel security upgrades [production]
14:32 <elukey> reboot analytics1003 for kernel + openjdk-8 upgrades [production]
14:28 <anomie@deploy1001> Synchronized php-1.32.0-wmf.20/includes/parser/ParserOutput.php: Backport for T203716 (duration: 00m 50s) [production]
14:26 <hashar> Switching CI Jenkins mail server from mx1001 to localhost | T203607 [production]
13:45 <marostegui> Drop user metrics and wikilytics from dbstore1002 [production]
12:23 <moritzm> restarting hhvm on mw1261-mw1265 to pick up lcms security update [production]
12:09 <moritzm> installing lcms2 security updates [production]
11:57 <moritzm> installing chromium on proton* (tested on deployment-prep with the new release) [production]
11:44 <volans> completed execution of "cookbook sre.switchdc.mediawiki --live-test codfw eqiad" - T199073 [production]
11:27 <END> (FAIL) - Cookbook sre.switchdc.mediawiki.08-restart-parsoid (exit_code=99) (volans@sarin) [production]
11:21 <START> - Cookbook sre.switchdc.mediawiki.08-restart-parsoid (volans@sarin) [production]
11:19 <END> (PASS) - Cookbook sre.switchdc.mediawiki.08-update-tendril (exit_code=0) (volans@sarin) [production]
11:18 <START> - Cookbook sre.switchdc.mediawiki.08-update-tendril (volans@sarin) [production]
11:16 <ladsgroup@deploy1001> Synchronized php-1.32.0-wmf.20/extensions/CentralAuth/maintenance/deleteLocalPasswords.php: [[gerrit:459504|SWAT: Fix typo (T201009)]] (duration: 00m 50s) [production]
11:14 <END> (PASS) - Cookbook sre.switchdc.mediawiki.08-start-maintenance (exit_code=0) (volans@sarin) [production]
11:13 <START> - Cookbook sre.switchdc.mediawiki.08-start-maintenance (volans@sarin) [production]
11:11 <END> (PASS) - Cookbook sre.switchdc.mediawiki.08-restore-ttl (exit_code=0) (volans@sarin) [production]
11:11 <START> - Cookbook sre.switchdc.mediawiki.08-restore-ttl (volans@sarin) [production]
11:09 <END> (PASS) - Cookbook sre.switchdc.mediawiki.07-set-readwrite (exit_code=0) (volans@sarin) [production]
11:09 <[DRY-RUN]> MediaWiki read-only period ends at: 2018-09-10 11:09:40.587760 (volans@sarin) [production]
11:09 <START> - Cookbook sre.switchdc.mediawiki.07-set-readwrite (volans@sarin) [production]
11:08 <moritzm> uploaded a co-installable 4.14 kernel to apt.wikimedia.org (to be used for installer tests) [production]
11:06 <ladsgroup@deploy1001> Synchronized wmf-config/CommonSettings.php: [[gerrit:458770|Add ['null'] (T201009)]] (duration: 00m 50s) [production]
10:57 <END> (PASS) - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite (exit_code=0) (volans@sarin) [production]
10:57 <START> - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite (volans@sarin) [production]
10:52 <END> (PASS) - Cookbook sre.switchdc.mediawiki.05-invert-redis-sessions (exit_code=0) (volans@sarin) [production]
10:52 <START> - Cookbook sre.switchdc.mediawiki.05-invert-redis-sessions (volans@sarin) [production]
10:50 <END> (PASS) - Cookbook sre.switchdc.mediawiki.04-switch-traffic (exit_code=0) (volans@sarin) [production]
10:48 <START> - Cookbook sre.switchdc.mediawiki.04-switch-traffic (volans@sarin) [production]
10:41 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:459523|Bumping portals to master (T128546)]] (duration: 00m 50s) [production]
10:40 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:459523|Bumping portals to master (T128546)]] (duration: 00m 50s) [production]