| 2013-10-07
      
      § | 
    
  | 18:55 | <reedy> | synchronized php-1.22wmf20/thumb.php | [production] | 
            
  | 18:40 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: commonswiki back to 1.22wmf19 | [production] | 
            
  | 18:38 | <ottomata> | backporting python-docopt 0.6.1 for precise and including in our apt repo | [production] | 
            
  | 18:24 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: Everything non 'pedia to 1.22wmf20 | [production] | 
            
  | 18:17 | <reedy> | synchronized php-1.22wmf20/extensions/Wikibase | [production] | 
            
  | 17:43 | <paravoid> | changed ms-fe1/2's mgmt IPs, ip clash with nas1-a/b's e0M | [production] | 
            
  | 17:28 | <mwalker> | payments update from I3a09af2cbd8566bbeb431437c49e72c832b87304 to I61bc80f257c590c1abeb716145c23921b005e5a8 | [production] | 
            
  | 16:05 | <krinkle> | synchronized wmf-config/InitialiseSettings.php  'I59e2547002bac5' | [production] | 
            
  | 14:50 | <RoanKattouw> | Restarting Parsoid on wtp10[01-24] on request; load avg reaching 90% | [production] | 
            
  | 14:28 | <Coren> | Labstore4 Configuring; will bounce up and down over the next two days. | [production] | 
            
  | 12:57 | <akosiaris> | brought icinga back up, after manually running puppet on neon | [production] | 
            
  | 12:39 | <hashar> | gallium : restarting Zuul. | [production] | 
            
  | 12:37 | <hashar> | gallium / Zuul : cherry picked a change from OpenStack related to statsd metrics. Our change: {{gerrit|88063}} | [production] | 
            
  | 11:30 | <mutante> | new public mailing list for the Wikimania Steering Commitee - wikimania-com | [production] | 
            
  | 11:25 | <hashar> | gallium : zuul restarted and apparently starting statsd metrics. | [production] | 
            
  | 11:03 | <hashar> | gallium : killed a bunch of stalled jenkins/java threads | [production] | 
            
  | 11:02 | <hashar> | gallium : restarting Zuul to enable statsd reporting {{gerrit|86744}} | [production] | 
            
  | 10:41 | <paravoid> | rebooting labstore3, NFS lockupo | [production] | 
            
  | 10:18 | <springle> | synchronized wmf-config/db-eqiad.php  'depool db1021' | [production] | 
            
  | 10:10 | <apergos> | powercycling srv291 | [production] | 
            
  | 09:44 | <springle> | synchronized wmf-config/db-eqiad.php  'warm up db1015' | [production] | 
            
  | 03:19 | <springle> | xtrabackup clone db43 to db1022 | [production] | 
            
  | 02:04 | <LocalisationUpdate> | failed: git pull of extensions failed | [production] | 
            
  | 00:59 | <springle> | synchronized wmf-config/db-eqiad.php  'depool db1022 for upgrade' | [production] | 
            
  
    | 2013-10-04
      
      § | 
    
  | 18:44 | <paravoid> | reinstalling erzurumi | [production] | 
            
  | 16:30 | <maxsem> | Finished syncing Wikimedia installation... : https://gerrit.wikimedia.org/r/87584 | [production] | 
            
  | 16:29 | <aaron> | synchronized php-1.22wmf20/includes/filerepo/file/LocalFile.php  'b48403885d5ac993cb3b3ce7ac7580002ab12e1f' | [production] | 
            
  | 16:15 | <maxsem> | Started syncing Wikimedia installation... : https://gerrit.wikimedia.org/r/87584 | [production] | 
            
  | 15:35 | <mark> | Added another 10G link to cr1-esams:ae1 <--> csw2-esams:ae1, now 40Gbps total | [production] | 
            
  | 15:03 | <mark> | Completely disconnected csw1-esams from the network | [production] | 
            
  | 12:12 | <mark> | Inserted 2x 10G MIC into cr1-esams FPC 0, brought online | [production] | 
            
  | 08:00 | <springle> | upgrading db1039 to precise + mariadb | [production] | 
            
  | 07:54 | <springle> | synchronized wmf-config/db-eqiad.php  'repool db1042' | [production] | 
            
  | 07:11 | <apergos> | powercycling maerlant, load > 100, couldn't log in via  mgmt console | [production] | 
            
  | 07:09 | <springle> | synchronized wmf-config/db-eqiad.php  'repool db1035 after upgrade, db1003 to full steam' | [production] | 
            
  | 06:23 | <springle> | synchronized wmf-config/db-eqiad.php  'repool db1003 after upgrade' | [production] | 
            
  | 06:14 | <Tim> | on fenari: running an analysis script for bug 53687 against tampa slave DB servers | [production] | 
            
  | 03:32 | <springle> | manual sync-common on mw1072 | [production] | 
            
  | 03:19 | <springle> | started xtrabackup clone from db1035 to db1003 | [production] | 
            
  | 02:42 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Fri Oct  4 02:41:40 UTC 2013 | [production] | 
            
  | 02:35 | <LocalisationUpdate> | completed (1.22wmf20) at Fri Oct  4 02:35:00 UTC 2013 | [production] | 
            
  | 02:31 | <springle> | synchronized wmf-config/db-eqiad.php  'refresh mw1072 db-eqiad.php' | [production] | 
            
  | 02:16 | <LocalisationUpdate> | completed (1.22wmf19) at Fri Oct  4 02:16:10 UTC 2013 | [production] | 
            
  | 02:15 | <springle> | upgrading db1003 to precise + mariadb | [production] |