| 2015-04-22
      
      § | 
    
  | 23:47 | <krenair> | Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/206015 (duration: 00m 12s) | [production] | 
            
  | 23:44 | <krenair> | Synchronized php-1.26wmf3/extensions/ZeroBanner/includes/ZeroSpecialPage.php: https://gerrit.wikimedia.org/r/#/c/206017/ (duration: 00m 13s) | [production] | 
            
  | 23:26 | <krenair> | Synchronized php-1.26wmf3/extensions/Flow: https://gerrit.wikimedia.org/r/#/c/206008/ (duration: 00m 13s) | [production] | 
            
  | 23:17 | <krenair> | Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/205889/ (duration: 00m 12s) | [production] | 
            
  | 23:16 | <krenair> | Synchronized php-1.26wmf2/extensions/OpenStackManager/nova/OpenStackNovaUser.php: https://gerrit.wikimedia.org/r/#/c/205887/ (duration: 00m 12s) | [production] | 
            
  | 22:55 | <twentyafterfour> | rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.26wmf3 | [production] | 
            
  | 22:52 | <twentyafterfour> | rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedias to 1.26wmf2 | [production] | 
            
  | 22:47 | <twentyafterfour> | Finished scap: testwiki to php-1.26wmf3 and rebuild l10n cache (duration: 37m 11s) | [production] | 
            
  | 22:44 | <hoo> | Killed demon's "sudo -u www-data php /srv/mediawiki-staging/multiversion/MWScript.php refreshLinks.php --wiki=ptwiki" on terbium, sending the box into swap | [production] | 
            
  | 22:10 | <twentyafterfour> | Started scap: testwiki to php-1.26wmf3 and rebuild l10n cache | [production] | 
            
  | 21:31 | <Coren> | reboot round of deployment-prep done | [production] | 
            
  | 21:05 | <Coren> | Starting deployment-prep rolling reboots | [production] | 
            
  | 20:10 | <twentyafterfour> | Started scap: testwiki to php-1.26wmf3 and rebuild l10n cache | [production] | 
            
  | 20:08 | <subbu> | deployed parsoid version 3311936a | [production] | 
            
  | 19:51 | <hashar> | Zuul / Jenkins back up and processing the 1+ hour backlog of changes.  Will take a while.  Multiple causes:  Zuul gearmand being stalled on a socket that has no more data to emit and  Jenkins being deadlocked due to an IRC plugin | [production] | 
            
  | 19:44 | <hashar> | Killing Jenkins cause .... we know | [production] | 
            
  | 19:27 | <hashar> | zuul gearman server is stalled | [production] | 
            
  | 15:30 | <gwicke> | stopped restbase on restbase1002 in preparation for cmjohnson1 checking the hardware | [production] | 
            
  | 15:30 | <demon> | Finished scap: 1.26wmf2 was tracking master. should be fixed, being paranoid and doing full sync + i18n rebuild (duration: 08m 11s) | [production] | 
            
  | 15:21 | <demon> | Started scap: 1.26wmf2 was tracking master. should be fixed, being paranoid and doing full sync + i18n rebuild | [production] | 
            
  | 15:19 | <demon> | Synchronized php-1.26wmf2/extensions/VisualEditor/: (no message) (duration: 00m 12s) | [production] | 
            
  | 15:19 | <demon> | Synchronized php-1.26wmf2/extensions/WikiEditor/: (no message) (duration: 00m 11s) | [production] | 
            
  | 15:12 | <demon> | Synchronized php-1.26wmf1/extensions/WikiEditor/: (no message) (duration: 00m 13s) | [production] | 
            
  | 13:37 | <godog> | ms-be101[678] weight to 2820 | [production] | 
            
  | 13:25 | <paravoid> | switched eqiad<->ulsfo link to Giglinx | [production] | 
            
  | 11:11 | <godog> | begin reimagining xenon, cerium and praseodymium | [production] | 
            
  | 07:39 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Wed Apr 22 07:38:22 UTC 2015 (duration 38m 21s) | [production] | 
            
  | 07:28 | <legoktm> | SULF is done, post-rename notifications are being sent out on the last large wikis | [production] | 
            
  | 03:20 | <ori> | Synchronized hhvm-fatal-error.php: I528e5384c: Increment a counter on fatals (duration: 00m 12s) | [production] | 
            
  | 02:56 | <LocalisationUpdate> | completed (1.26wmf2) at 2015-04-22 02:55:44+00:00 | [production] | 
            
  | 02:50 | <l10nupdate> | Synchronized php-1.26wmf2/cache/l10n: (no message) (duration: 08m 31s) | [production] | 
            
  | 02:26 | <LocalisationUpdate> | completed (1.26wmf1) at 2015-04-22 02:25:40+00:00 | [production] | 
            
  | 02:22 | <l10nupdate> | Synchronized php-1.26wmf1/cache/l10n: (no message) (duration: 05m 45s) | [production] | 
            
  | 01:50 | <springle> | Synchronized wmf-config/db-eqiad.php: repool db1019, warm up (duration: 00m 13s) | [production] | 
            
  
    | 2015-04-21
      
      § | 
    
  | 23:07 | <krenair> | Synchronized wmf-config/throttle.php: https://gerrit.wikimedia.org/r/#/c/205640/ (duration: 00m 13s) | [production] | 
            
  | 23:04 | <krenair> | Synchronized php-1.26wmf2/extensions/VisualEditor: https://gerrit.wikimedia.org/r/205774 - should effectively be a no-op until config (duration: 00m 12s) | [production] | 
            
  | 22:24 | <robh> | disabled a bunch of old rt queues from allowing ticket creation, tired of spam | [production] | 
            
  | 20:53 | <aaron> | Synchronized php-1.26wmf1/includes/jobqueue/JobRunner.php: 4285f1921585ee87034e9739b1353fbad35f3a29 (duration: 00m 11s) | [production] | 
            
  | 20:53 | <aaron> | Synchronized php-1.26wmf1/includes/GlobalFunctions.php: bceb4de391bd8a321921a8587988cb1be7b71556 (duration: 00m 11s) | [production] | 
            
  | 20:34 | <aaron> | Synchronized php-1.26wmf2/includes/jobqueue/JobRunner.php: 2f3b7594650162b04f55e63e8df251d3913ab7ca (duration: 00m 11s) | [production] | 
            
  | 18:24 | <twentyafterfour> | rebuilt wikiversions.cdb and synchronized wikiversions files: group1 to 1.26wmf2 | [production] | 
            
  | 17:49 | <aude> | Synchronized wmf-config/Wikibase.php: Add subscriptionLookupMode setting for wikidata (duration: 00m 13s) | [production] | 
            
  | 17:32 | <aaron> | Synchronized php-1.26wmf2/includes/GlobalFunctions.php: b5b054e2f5b53e30d5aca21d046aa0ac33d5c407 (duration: 00m 12s) | [production] | 
            
  | 16:41 | <legoktm> | Synchronized php-1.26wmf1/extensions/OAI/OAIHooks.php: Don't try to update up_page=0 if page moves suppressed redirects (duration: 00m 13s) | [production] | 
            
  | 16:40 | <legoktm> | Synchronized php-1.26wmf2/extensions/OAI/OAIHooks.php: Don't try to update up_page=0 if page moves suppressed redirects (duration: 00m 11s) | [production] | 
            
  | 15:45 | <legoktm> | Synchronized php-1.26wmf2/extensions/OAI/OAIHooks.php: better debugging for T96686 (duration: 00m 11s) | [production] | 
            
  | 15:11 | <thcipriani> | Synchronized wmf-config/InitialiseSettings.php: SWAT [[gerrit:204722]] (duration: 00m 11s) | [production] | 
            
  | 14:15 | <legoktm> | Synchronized php-1.26wmf1/extensions/OAI/OAIHooks.php: better debugging for T96686 (duration: 00m 11s) | [production] | 
            
  | 14:15 | <springle> | enwiki master under unusual jobrunner load, not terminal but see https://phabricator.wikimedia.org/T96686 | [production] | 
            
  | 08:47 | <ori> | Synchronized php-1.26wmf1/includes/jobqueue: Ifa478996f: Revert 'Added per-wiki queue stats information' (duration: 00m 12s) | [production] |