| 2016-02-04
      
      § | 
    
  | 23:54 | <mobrovac> | restbase disabled temporarily puppet in prod to test https://gerrit.wikimedia.org/r/#/c/268597/ | [production] | 
            
  | 23:29 | <mutante> | rcs1001 - rebooting | [production] | 
            
  | 22:45 | <bblack> | rebooting cp1060 to test traffic-pool stuff | [production] | 
            
  | 22:30 | <demon@mira> | Synchronized wmf-config/: undo my cleanup grumble grumble (duration: 01m 16s) | [production] | 
            
  | 22:12 | <demon@mira> | Synchronized wmf-config/: touch (duration: 01m 14s) | [production] | 
            
  | 22:10 | <demon@mira> | Synchronized private/: touch (duration: 01m 15s) | [production] | 
            
  | 22:01 | <demon@mira> | Synchronized wmf-config/PrivateSettings.php: touch symlink (duration: 01m 15s) | [production] | 
            
  | 21:57 | <demon@mira> | Synchronized wmf-config/: gerrit 268471, 268454 (duration: 01m 18s) | [production] | 
            
  | 21:52 | <mutante> | mira chgrp -R wikidev /srv/mediawiki-staging/.git/objects | [production] | 
            
  | 21:29 | <mutante> | rcs1001 started redis | [production] | 
            
  | 21:21 | <paravoid> | setting up OSPF/OSPF3/PIM between ulsfo and codfw (cr2-ulsfo/cr1-codfw) | [production] | 
            
  | 21:19 | <mutante> | rcs1002 - start redis | [production] | 
            
  | 21:15 | <mutante> | rcs1002 - reboot for kernel | [production] | 
            
  | 20:45 | <mutante_> | rsc1001 - schedule downtime, reboot | [production] | 
            
  | 20:30 | <paravoid> | cr1-ulsfo: deactivating BGP peering with GTT | [production] | 
            
  | 20:26 | <hashar> | All wikis to 1.27.0-wmf.12 No troubles so far congratulations to everyone involved @wikimedia #wikimedia | [production] | 
            
  | 20:26 | <mutante> | eeden service ntp restart | [production] | 
            
  | 20:23 | <mutante> | mw1115 service hhvm restart | [production] | 
            
  | 20:16 | <mutante> | mw1117 - powercycled | [production] | 
            
  | 20:15 | <paravoid> | cr1-ulsfo: turning up BGP with Zayo | [production] | 
            
  | 20:15 | <mutante> | scb1001/scb1002 service mathoid restart | [production] | 
            
  | 20:03 | <hashar@mira> | rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.27.0-wmf.12 | [production] | 
            
  | 20:03 | <hashar> | all wikis to 1.27.0-wmf.12 (yeah really) | [production] | 
            
  | 20:00 | <mobrovac> | restbase rolling restart after merging https://gerrit.wikimedia.org/r/#/c/268016/ | [production] | 
            
  | 19:54 | <demon@mira> | Synchronized private/PrivateSettings.php: (no message) (duration: 02m 05s) | [production] | 
            
  | 19:43 | <ottomata1> | rebooting kafka1012 | [production] | 
            
  | 19:32 | <demon@mira> | Synchronized wmf-config/InitialiseSettings.php: T125850 (duration: 02m 11s) | [production] | 
            
  | 19:00 | <demon@mira> | Synchronized wmf-config/InitialiseSettings-labs.php: prod no op for completeness (duration: 03m 02s) | [production] | 
            
  | 18:49 | <demon@mira> | Synchronized wmf-config/: removing old mwblocker.log (duration: 02m 07s) | [production] | 
            
  | 18:45 | <demon@mira> | Synchronized private/mwblocker.log: (no message) (duration: 02m 10s) | [production] | 
            
  | 18:40 | <yurik> | deployed and reenabled tilerator & tileratorui | [production] | 
            
  | 17:30 | <hoo@mira> | Synchronized php-1.27.0-wmf.12/extensions/Wikidata: Fix editing terms in languages other than the interface language via the term box (duration: 02m 18s) | [production] | 
            
  | 17:29 | <hoo> | (Re)started wdqs-updater on wdqs1001, but seems it doesn't work | [production] | 
            
  | 16:53 | <twentyafterfour> | restarted phd to synchronize settings with phabricator | [production] | 
            
  | 16:52 | <twentyafterfour> | restarted apache2 on iridium so that phabricator recognizes sprint.phragile-uri | [production] | 
            
  | 16:45 | <thcipriani@mira> | Synchronized php-1.27.0-wmf.12/includes/media/Bitmap.php: SWAT: BitmapHandler: Implement validateParam() [[gerrit:268407]] (duration: 02m 08s) | [production] | 
            
  | 16:07 | <thcipriani@mira> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable confirmed group at nowiki [[gerrit:267804]] (duration: 02m 15s) | [production] | 
            
  | 15:53 | <urandom> | rolling restbase restart complete | [production] | 
            
  | 15:47 | <urandom> | restbase cluster puppet run complete; performing rolling restart of restbase (applying https://gerrit.wikimedia.org/r/#/c/266297/) | [production] | 
            
  | 15:44 | <moritzm> | rebooting db203[678] for kernel update | [production] | 
            
  | 15:35 | <urandom> | forcing puppet run on restbase cluster (config deploy) | [production] | 
            
  | 15:34 | <urandom> | reenabling puppet on restbase cluster (continue config deploy) | [production] | 
            
  | 15:18 | <urandom> | restarting restbase on restbase1001.eqiad.wmnet (config deploy) | [production] | 
            
  | 15:15 | <urandom> | re-enabling puppet and forcing run on restbase1001.eqiad.wmnet (canary config deploy) | [production] | 
            
  | 15:11 | <_joe_> | restarting pybal on lvs200{3,6} | [production] | 
            
  | 15:08 | <urandom> | disabling puppet on restbase cluster in preparation for configuration deploy (https://gerrit.wikimedia.org/r/#/c/266297/) | [production] | 
            
  | 15:07 | <moritzm> | rebooting oxygen for kernel update | [production] | 
            
  | 14:58 | <ottomata> | stopping eventlogging to reboot eventlog1001 for kernel update | [production] |