701-750 of 10000 results (29ms)
2016-04-22 §
02:31 <l10nupdate@tin> ResourceLoader cache refresh completed at Fri Apr 22 02:31:45 UTC 2016 (duration 9m 15s) [production]
02:22 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.21) (duration: 10m 12s) [production]
2016-04-21 §
20:34 <gehel> lowering elasticsearch high watermark to rebalance disk space across cluster [production]
20:07 <gehel> reindex elasticsearch updates for the duration of the switch back from codfw, just in case [production]
18:57 <mutante> rutherfordium - on 4.4.0-1-amd64 now (T131928) [production]
18:56 <mutante> people.wm.org / rutherfordium , very short downtime, reboot [production]
18:43 <mutante> rutherfordium (people.wm.org) - installing package upgrades [production]
18:29 <mutante> bromine - on 4.4.0-1-amd64 now (T131928) [production]
18:25 <mutante> bromine (annualreport,bz-static,transparency,releases) reboot for kernel upgrade [production]
18:15 <mutante> bromine - apt-get dist-upgrade [production]
18:12 <mutante> planet1001 - on 4.4.0-1-amd64 now (T131928) [production]
18:10 <mutante> planet1001 - reboot for upgrade [production]
17:44 <mutante> planet1001 - apt-get dist-upgrade (libc6, apache, ..) [production]
16:10 <marktraceur@tin> Synchronized php-1.27.0-wmf.21/resources/src/mediawiki/api/upload.js: Unbreak finishing stash uploads in upload dialog (duration: 00m 27s) [production]
16:10 <jynus> disabling event scheduler on db1041 [production]
15:34 <_joe_> services switchover done [production]
15:30 <volans@tin> Synchronized wmf-config/db-eqiad.php: Adust weights for shard s7 - T133205 (duration: 00m 32s) [production]
15:28 <_joe_> [switchback services] moving traffic for restbase/citoid/cxserver back to eqiad [production]
15:23 <ostriches> ytterbium: puppet re-enabled for gerrit host [production]
15:23 <filippo@tin> Synchronized wmf-config/filebackend-production.php: [switchover swift #1] async writes to codfw (duration: 00m 28s) [production]
15:15 <godog> [switchover swift #2] upload backends from codfw to eqiad [production]
15:09 <godog> [switchover swift #3] upload codfw to eqiad [production]
15:05 <godog> [switchover swift #4] repool upload eqiad in dns [production]
15:04 <gehel> starting reindex of lost elasticsearch updates during activation of SSL (T132762) [production]
15:01 <godog> [switchover swift #5] upload esams to eqiad [production]
14:57 <godog> [switchover swift #6] upload eqiad to 'direct' [production]
14:48 <gehel@tin> Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 29s) [production]
14:45 <gehel> reverting elasticsearch traffic back to eqiad [production]
14:39 <jynus> enabling replication lag alerts for all dbs [production]
14:28 <paravoid> traffic/mediawiki codfw->eqiad switchover is done [production]
14:26 <jynus> deployed dns updated records for new eqiad masters [production]
14:25 <_joe_> [switchover #8/#3] Running rebuildEntityPerPage.php [production]
14:25 <_joe_> [switchover #8/#2] Enabling crons in eqiad [production]
14:24 <ori> [switchover #8/#1] Starting the jobqueue in eqiad [production]
14:23 <volans> [switchover #8/#3] Re-enable puppet on all eqiad and codfw databases masters [production]
14:21 <paravoid> wikis are read-write again [production]
14:21 <ori@tin> Synchronized wmf-config/db-eqiad.php: [switchover #7/#1] Iac92c8bc6b: Put eqiad in read-write mode for datacenter switchover to eqiad (duration: 00m 39s) [production]
14:20 <volans> [switchover #6/#1] Set database masters RW in eqiad for s1-7, es2-3 and x1 [production]
14:19 <bblack> [switchover #5/#6] Switch Varnish MW backends to eqiad - DONE (confirmed) [production]
14:18 <Krinkle> apache-fast-test warmup finished [production]
14:18 <godog> [switchover #5/#7] finished swift-proxy roll restart [production]
14:15 <bblack> [switchover #5/#6] Switch Varnish MW backends to eqiad - starting [production]
14:14 <mobrovac> [switchover #5/2] restbase {{done}} [production]
14:11 <subbu> parsoid deploy and restarts done [production]
14:10 <godog> [switchover #5/#7] roll-restart swift-proxy in eqiad and codfw [production]
14:09 <subbu> restarting parsoid on all nodes [production]
14:09 <akosiaris> [switchover #5/#3] Misc services cluster (for the action API endpoint) [production]
14:09 <mobrovac> [switchover #5/2] restbase puppet agent -tv && systemctl restart restbase [production]
14:08 <Krinkle> krinkle@tin: bin/apache-fast-test wiki-urls-warmup1000.txt eqiad [production]
14:08 <subbu> syncing parsoid code [production]