| 2016-05-31
      
      § | 
    
  | 21:33 | <urandom> | Bouncing restbase on restbase1015.eqiad.wmnet | [production] | 
            
  | 21:32 | <urandom> | Bouncing restbase on restbase1013.eqiad.wmnet | [production] | 
            
  | 21:31 | <urandom> | Bouncing restbase on restbase1012.eqiad.wmnet | [production] | 
            
  | 21:30 | <urandom> | Bouncing restbase on restbase1009.eqiad.wmnet | [production] | 
            
  | 21:29 | <urandom> | Bouncing restbase on restbase1008.eqiad.wmnet | [production] | 
            
  | 21:28 | <urandom> | Bouncing restbase on restbase1010.eqiad | [production] | 
            
  | 20:58 | <thcipriani@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group0 back to 1.28.0-wmf.3 | [production] | 
            
  | 20:55 | <thcipriani> | rolling back group0 wmf.4 for T136644 too much log spam | [production] | 
            
  | 20:39 | <thcipriani@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group0 to 1.28.0-wmf.4 | [production] | 
            
  | 20:35 | <thcipriani@tin> | Synchronized php-1.28.0-wmf.3/extensions/Wikidata/extensions/Wikibase/view/resources/jquery/ui/jquery.ui.tagadata.js: [[gerit:291974|Update Wikibase]] (duration: 00m 23s) | [production] | 
            
  | 20:33 | <thcipriani@tin> | Synchronized php-1.28.0-wmf.4/extensions/Wikidata/extensions/Wikibase/view/resources/jquery/ui/jquery.ui.tagadata.js: [[gerit:291974|Update Wikibase]] (duration: 00m 30s) | [production] | 
            
  | 20:28 | <bblack> | depooled reboot of cp3043 - T126062 | [production] | 
            
  | 20:23 | <bblack> | depooled reboot of cp3042 - T126062 | [production] | 
            
  | 20:13 | <thcipriani@tin> | Finished scap: testwiki to php-1.28.0-wmf.4 and rebuild l10n cache (duration: 50m 09s) | [production] | 
            
  | 20:10 | <urandom> | disabling cql binary transport on restbase1007-c | [production] | 
            
  | 20:09 | <bblack> | depooled reboot of cp3041 - T126062 | [production] | 
            
  | 20:02 | <bblack> | depooled reboot of cp3033 (not 3032) - T126062 | [production] | 
            
  | 20:02 | <bblack> | depooled reboot of cp3032 - T126062 | [production] | 
            
  | 19:57 | <bblack> | depooled reboot of cp3031 - T126062 | [production] | 
            
  | 19:50 | <bblack> | depooled reboot of cp3030 - T126062 | [production] | 
            
  | 19:45 | <bblack> | restarting cp* nginxes for config update | [production] | 
            
  | 19:23 | <thcipriani@tin> | Started scap: testwiki to php-1.28.0-wmf.4 and rebuild l10n cache | [production] | 
            
  | 19:14 | <urandom> | Restart restbase1007-c.eqiad.wmnet because reasons | [production] | 
            
  | 18:13 | <Krinkle> | mwscript deleteEqualMessages.php --wiki hrwikibooks (T45917) | [production] | 
            
  | 18:07 | <ejegg> | updated payments-wiki from e6807395d7687d521070b83d159b77b242e5c04f to 5bb160e9898224e1d7d0a5c57fe408edb998a262 | [production] | 
            
  | 17:26 | <jynus> | restarting mysqls at sanitarium, some transitional lag on labs | [production] | 
            
  | 17:17 | <bblack> | depooled reboot of cp3040 - T126062 | [production] | 
            
  | 17:03 | <thcipriani> | starting branch-cut for mediawiki and extensions for version 1.28.0-wmf.4 | [production] | 
            
  | 16:52 | <elukey> | disabling puppet on mc10* hosts as prep step for https://gerrit.wikimedia.org/r/#/c/291916. Memcached 1.4.25 will be deployed to mc1010 as part of a perf. test (T129963) | [production] | 
            
  | 16:15 | <urandom> | Upgrade of restbase1007-{a,b,c} complete : T126629 | [production] | 
            
  | 16:10 | <urandom> | Upgrading Cassandra to 2.2.6 on restbase1007.eqiad.wmnet : T126629 | [production] | 
            
  | 16:07 | <bblack> | depooling cp3032 to investigate T126062 | [production] | 
            
  | 16:06 | <urandom> | Stopping restbase1007-{a,b,c} in preparation for upgrade : T126629 | [production] | 
            
  | 15:51 | <urandom> | Disabling puppet in preparation for upgrade on restbase1007, 1010, and 1011 : T126629 | [production] | 
            
  | 15:10 | <thcipriani@tin> | Synchronized php-1.28.0-wmf.3/extensions/EventBus/EventBus.hooks.php: SWAT: [[gerrit:291904|Replace wfUrlEncode with rawurlencode]] (duration: 00m 27s) | [production] | 
            
  | 15:07 | <moritzm> | rebooting mendelevium (ticket.wikimedia.org) for update to Linux 4.4 | [production] | 
            
  | 15:05 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:291766|Math: Enable MathML everywhere but private wikis]] (duration: 00m 34s) | [production] | 
            
  | 15:03 | <bblack> | restarting all frontend caches for new memory params (randomized order, ~1-min spacing, ~2h to completion) - T135384 | [production] | 
            
  | 14:54 | <moritzm> | rolling reboot of scb systems in codfw for Linux 4.4 upgrade | [production] | 
            
  | 12:21 | <moritzm> | continue rolling reboot of mc2* systems for Linux 4.4 upgrade | [production] | 
            
  | 08:50 | <mobrovac> | restbase deploy end of fcd62e1 | [production] | 
            
  | 08:34 | <mobrovac> | restbase deploy start of fcd62e1 | [production] | 
            
  | 08:04 | <jynus> | db1049> megacli -PDOnline -PhysDrv '[32:4]' -a0 | [production] | 
            
  | 07:46 | <mobrovac> | change-prop deploying 980f65c | [production] | 
            
  | 07:15 | <jynus> | db1049> megacli -PDOffline -PhysDrv '[32:4]' -a0 | [production] | 
            
  | 06:20 | <moritzm> | upgrading hhvm in eqiad (also picking up updated versions of icu and lcms) | [production] | 
            
  | 02:31 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Tue May 31 02:31:36 UTC 2016 (duration 5m 54s) | [production] | 
            
  | 02:25 | <mwdeploy@tin> | scap sync-l10n completed (1.28.0-wmf.3) (duration: 09m 30s) | [production] |