| 
      
        2016-02-05
      
      §
     | 
  
    
  | 12:38 | 
  <elukey> | 
  repooled mw1228.eqiad.wmnet   | 
  [production] | 
            
  | 12:34 | 
  <moritzm> | 
  rebooting db2050 to db2054 for kernel update | 
  [production] | 
            
  | 12:15 | 
  <moritzm> | 
  rebooting db2045 to db2049 for kernel update | 
  [production] | 
            
  | 12:07 | 
  <jynus> | 
  reimporting nlwiktionary pages into labs | 
  [production] | 
            
  | 12:05 | 
  <l10nupdate@tin> | 
  LocalisationUpdate failed: git pull of core failed | 
  [production] | 
            
  | 12:05 | 
  <l10nupdate@tin> | 
  LocalisationUpdate failed: git clone of core failed | 
  [production] | 
            
  | 11:54 | 
  <moritzm> | 
  rebooting db2041 to db2044 for kernel update | 
  [production] | 
            
  | 11:37 | 
  <moritzm> | 
  rebooting db2038 to db2040 for kernel update | 
  [production] | 
            
  | 11:36 | 
  <godog> | 
  start swiftrepl replication pass of common thumbs eqiad -> codfw | 
  [production] | 
            
  | 10:15 | 
  <moritzm> | 
  rolling reboot of ocg* cluster | 
  [production] | 
            
  | 02:27 | 
  <mobrovac> | 
  restbase deploy end of caae1f7 | 
  [production] | 
            
  | 02:20 | 
  <mobrovac> | 
  restbase deploy start of caae1f7 | 
  [production] | 
            
  | 01:52 | 
  <TimStarling> | 
  deploying apache log format change following successful test on deployment-prep | 
  [production] | 
            
  | 01:47 | 
  <aude@mira> | 
  Finished scap: Re-add user rights messages in Echo (duration: 24m 37s) | 
  [production] | 
            
  | 01:22 | 
  <aude@mira> | 
  Started scap: Re-add user rights messages in Echo | 
  [production] | 
            
  | 01:20 | 
  <aude@mira> | 
  Synchronized php-1.27.0-wmf.12/extensions/Echo: Re-add user rights messages in Echo (duration: 01m 20s) | 
  [production] | 
            
  | 01:06 | 
  <aude@mira> | 
  Synchronized wmf-config/: Sync wikidata config changes for beta (duration: 01m 15s) | 
  [production] | 
            
  | 01:00 | 
  <mobrovac> | 
  restbase deploy end of 2aef1b67a0 | 
  [production] | 
            
  | 00:58 | 
  <aude@mira> | 
  Synchronized wmf-config/InitialiseSettings.php: Set $wgEnotifMinorEdits to true on huwiki (duration: 01m 16s) | 
  [production] | 
            
  | 00:52 | 
  <aude@mira> | 
  Synchronized wmf-config/InitialiseSettings.php: Add museumvictoria.com.au to $wgCopyUploadsDomains (duration: 01m 17s) | 
  [production] | 
            
  | 00:46 | 
  <aude@mira> | 
  Synchronized wmf-config/InitialiseSettings.php: Re-enable ShortUrl on maiwiki, bhwiki and orwikisource, after creating db table (duration: 01m 18s) | 
  [production] | 
            
  | 00:36 | 
  <aude@mira> | 
  Synchronized wmf-config/InitialiseSettings.php: revert shorturl changes (duration: 01m 17s) | 
  [production] | 
            
  | 00:29 | 
  <aude@mira> | 
  Synchronized wmf-config/InitialiseSettings.php: Enable ShortUrl on maiwiki, bhwiki and orwikisource (duration: 01m 17s) | 
  [production] | 
            
  | 00:26 | 
  <akosiaris> | 
  restart apache on mendelevium.eqiad.wmnet. seems there's a memory leak, need to investigate tomorrow | 
  [production] | 
            
  | 00:23 | 
  <aude@mira> | 
  Synchronized wmf-config/InitialiseSettings.php: Re-enable category watch on wikipedia and commons (duration: 01m 18s) | 
  [production] | 
            
  | 00:19 | 
  <mobrovac> | 
  restbase deploy start of 2aef1b67a0 on rb1001 | 
  [production] | 
            
  
    | 
      
        2016-02-04
      
      §
     | 
  
    
  | 23:54 | 
  <mobrovac> | 
  restbase disabled temporarily puppet in prod to test https://gerrit.wikimedia.org/r/#/c/268597/ | 
  [production] | 
            
  | 23:29 | 
  <mutante> | 
  rcs1001 - rebooting | 
  [production] | 
            
  | 22:45 | 
  <bblack> | 
  rebooting cp1060 to test traffic-pool stuff | 
  [production] | 
            
  | 22:30 | 
  <demon@mira> | 
  Synchronized wmf-config/: undo my cleanup grumble grumble (duration: 01m 16s) | 
  [production] | 
            
  | 22:12 | 
  <demon@mira> | 
  Synchronized wmf-config/: touch (duration: 01m 14s) | 
  [production] | 
            
  | 22:10 | 
  <demon@mira> | 
  Synchronized private/: touch (duration: 01m 15s) | 
  [production] | 
            
  | 22:01 | 
  <demon@mira> | 
  Synchronized wmf-config/PrivateSettings.php: touch symlink (duration: 01m 15s) | 
  [production] | 
            
  | 21:57 | 
  <demon@mira> | 
  Synchronized wmf-config/: gerrit 268471, 268454 (duration: 01m 18s) | 
  [production] | 
            
  | 21:52 | 
  <mutante> | 
  mira chgrp -R wikidev /srv/mediawiki-staging/.git/objects | 
  [production] | 
            
  | 21:29 | 
  <mutante> | 
  rcs1001 started redis | 
  [production] | 
            
  | 21:21 | 
  <paravoid> | 
  setting up OSPF/OSPF3/PIM between ulsfo and codfw (cr2-ulsfo/cr1-codfw) | 
  [production] | 
            
  | 21:19 | 
  <mutante> | 
  rcs1002 - start redis | 
  [production] | 
            
  | 21:15 | 
  <mutante> | 
  rcs1002 - reboot for kernel | 
  [production] | 
            
  | 20:45 | 
  <mutante_> | 
  rsc1001 - schedule downtime, reboot | 
  [production] | 
            
  | 20:30 | 
  <paravoid> | 
  cr1-ulsfo: deactivating BGP peering with GTT | 
  [production] | 
            
  | 20:26 | 
  <hashar> | 
  All wikis to 1.27.0-wmf.12 No troubles so far congratulations to everyone involved @wikimedia #wikimedia | 
  [production] | 
            
  | 20:26 | 
  <mutante> | 
  eeden service ntp restart | 
  [production] | 
            
  | 20:23 | 
  <mutante> | 
  mw1115 service hhvm restart | 
  [production] | 
            
  | 20:16 | 
  <mutante> | 
  mw1117 - powercycled | 
  [production] | 
            
  | 20:15 | 
  <paravoid> | 
  cr1-ulsfo: turning up BGP with Zayo | 
  [production] | 
            
  | 20:15 | 
  <mutante> | 
  scb1001/scb1002 service mathoid restart | 
  [production] | 
            
  | 20:03 | 
  <hashar@mira> | 
  rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.27.0-wmf.12 | 
  [production] | 
            
  | 20:03 | 
  <hashar> | 
  all wikis to 1.27.0-wmf.12 (yeah really) | 
  [production] | 
            
  | 20:00 | 
  <mobrovac> | 
  restbase rolling restart after merging https://gerrit.wikimedia.org/r/#/c/268016/ | 
  [production] |