| 2015-01-09
      
      § | 
    
  | 01:49 | <maxsem> | Synchronized php-1.25wmf13/extensions/MobileFrontend: touch (duration: 00m 09s) | [production] | 
            
  | 01:25 | <maxsem> | Finished scap: SWAT: MobileFrontend and WikiGrok updates (duration: 17m 36s) | [production] | 
            
  | 01:07 | <maxsem> | Started scap: SWAT: MobileFrontend and WikiGrok updates | [production] | 
            
  | 00:53 | <hoo> | Updated the Wikidata property suggester with data from Monday's JSON dump | [production] | 
            
  | 00:46 | <legoktm> | Synchronized wmf-config/InitialiseSettings.php: SWAT: https://gerrit.wikimedia.org/r/#/c/180451/ (duration: 00m 06s) | [production] | 
            
  | 00:45 | <legoktm> | Synchronized closed.dblist: SWAT: https://gerrit.wikimedia.org/r/#/c/180451/ (duration: 00m 08s) | [production] | 
            
  | 00:42 | <legoktm> | Synchronized php-1.25wmf13/extensions/CentralAuth/: SWAT: https://gerrit.wikimedia.org/r/#/c/183554/ (duration: 00m 06s) | [production] | 
            
  | 00:40 | <legoktm> | Synchronized php-1.25wmf14/extensions/CentralAuth/: SWAT: https://gerrit.wikimedia.org/r/#/c/183554/ (duration: 00m 06s) | [production] | 
            
  | 00:37 | <legoktm> | Synchronized php-1.25wmf13/extensions/Scribunto/engines/LuaCommon/lualib/mw.title.lua: SWAT: https://gerrit.wikimedia.org/r/#/c/183552/ again (duration: 00m 07s) | [production] | 
            
  | 00:36 | <legoktm> | Synchronized php-1.25wmf14/extensions/Scribunto/engines/LuaCommon/lualib/mw.title.lua: SWAT: https://gerrit.wikimedia.org/r/#/c/183552/ again (duration: 00m 06s) | [production] | 
            
  | 00:29 | <legoktm> | Synchronized php-1.25wmf13/extensions/Scribunto/engines/LuaCommon/lualib/mw.title.lua: SWAT: https://gerrit.wikimedia.org/r/#/c/183552/ (duration: 00m 07s) | [production] | 
            
  | 00:27 | <legoktm> | Synchronized php-1.25wmf14/extensions/Scribunto/engines/LuaCommon/lualib/mw.title.lua: SWAT: https://gerrit.wikimedia.org/r/#/c/183552/ (duration: 00m 06s) | [production] | 
            
  
    | 2015-01-08
      
      § | 
    
  | 23:46 | <ori> | Synchronized wmf-config/mc.php: (no message) (duration: 00m 07s) | [production] | 
            
  | 23:32 | <ori> | Synchronized wmf-config/mc.php: Revert: I4c4691e26: memcached: use a unix socket instead of a tcp connection on selected hosts (duration: 00m 06s) | [production] | 
            
  | 23:30 | <ori> | Synchronized wmf-config/mc.php: I4c4691e26: memcached: use a unix socket instead of a tcp connection on selected hosts (duration: 00m 06s) | [production] | 
            
  | 23:26 | <ori> | depooling mw1230 and mw1231 for a couple of minutes for I4c4691e26 | [production] | 
            
  | 21:53 | <reedy> | Synchronized wmf-config/InitialiseSettings.php: nooop to test scap update (duration: 00m 06s) | [production] | 
            
  | 21:52 | <mutante> | fixing scap permissions on mediawiki-installation servers via dsh | [production] | 
            
  | 21:29 | <reedy> | Synchronized wmf-config/InitialiseSettings.php: nooop to test scap update (duration: 00m 06s) | [production] | 
            
  | 21:04 | <reedy> | Synchronized wmf-config/InitialiseSettings.php: nooop to test scap update (duration: 00m 09s) | [production] | 
            
  | 21:00 | <reedy> | Synchronized wmf-config/InitialiseSettings.php: nooop to test scap update (duration: 00m 06s) | [production] | 
            
  | 20:39 | <Reedy> | Scap deployed at a78ddec | [production] | 
            
  | 20:22 | <Reedy> | moved /srv/deployment/scap to scap.old as git repo seems busted. Hoping puppet puts it back again correctly... | [production] | 
            
  | 19:47 | <Reedy> | scap-rebuild-cdbs finished | [production] | 
            
  | 19:44 | <Reedy> | running dsh -g mediawiki-installation -M -F 40 -- "sudo -u mwdeploy /srv/deployment/scap/scap/bin/scap-rebuild-cdbs" | [production] | 
            
  | 19:43 | <reedy> | Synchronized php-1.25wmf14/cache/l10n/: l10nupdate (duration: 03m 39s) | [production] | 
            
  | 19:39 | <reedy> | Synchronized php-1.25wmf13/cache/l10n/: (no message) (duration: 00m 05s) | [production] | 
            
  | 19:38 | <reedy> | Synchronized php-1.25wmf13/cache/l10n/: l10nupdate (duration: 03m 54s) | [production] | 
            
  | 19:26 | <LocalisationUpdate> | completed (1.25wmf14) at 2015-01-08 19:26:46+00:00 | [production] | 
            
  | 19:25 | <reedy> | Synchronized php-1.25wmf14/cache/l10n: (no message) (duration: 00m 01s) | [production] | 
            
  | 19:14 | <LocalisationUpdate> | completed (1.25wmf13) at 2015-01-08 19:14:37+00:00 | [production] | 
            
  | 19:13 | <reedy> | Synchronized php-1.25wmf13/cache/l10n: (no message) (duration: 00m 01s) | [production] | 
            
  | 18:58 | <Reedy> | Attempting manual run of l10nupdate | [production] | 
            
  | 17:08 | <marktraceur> | Finished scap: [SWAT] [AbuseFilter] Add file_size variable (duration: 33m 27s) | [production] | 
            
  | 16:51 | <hashar> | Restarted Zuul. Same issue as https://wikitech.wikimedia.org/wiki/Incident_documentation/20150106-Zuul | [production] | 
            
  | 16:35 | <marktraceur> | Started scap: [SWAT] [AbuseFilter] Add file_size variable | [production] | 
            
  | 16:25 | <marktraceur> | Synchronized php-1.25wmf14/extensions/AbuseFilter: [SWAT] [AbuseFilter] Add file_size variable (duration: 00m 06s) | [production] | 
            
  | 16:14 | <marktraceur> | Synchronized php-1.25wmf14/extensions/VisualEditor/: [SWAT] VisualEditor fixes for T86046 and T86056 (duration: 00m 05s) | [production] | 
            
  | 16:02 | <marktraceur> | Synchronized wmf-config/logging.php: [SWAT] Honor log sampling and levels for logstash on group0 wikis (duration: 00m 05s) | [production] | 
            
  | 15:44 | <_joe_> | restarting hhvm on mw1226, TC full | [production] | 
            
  | 15:37 | <_joe_> | restarting hhvm on mw1113, stuck in parsing the ini file (HPHP::is_valid_var_name) | [production] | 
            
  | 15:08 | <aude> | Synchronized wmf-config/Wikibase.php: Bump cache epoch for test.wikidata (duration: 00m 06s) | [production] | 
            
  | 15:01 | <aude> | Finished scap: Update group0 to wmf/1.25wmf14 Wikidata extension branch (duration: 26m 43s) | [production] | 
            
  | 14:35 | <aude> | Started scap: Update group0 to wmf/1.25wmf14 Wikidata extension branch | [production] | 
            
  | 14:34 | <Jeff_Green> | samarium package updates and reboot | [production] | 
            
  | 11:37 | <godog> | reboot ms-be1011, xfs hosed :( | [production] | 
            
  | 11:07 | <_joe_> | repooled the canary servers | [production] | 
            
  | 10:55 | <_joe_> | installing a new hhvm package (with the correct libicu dependence) on canary hosts | [production] | 
            
  | 10:02 | <godog> | removing backend hosts from LVS for search pools | [production] | 
            
  | 09:42 | <godog> | restart uwsgi on tungsten | [production] |