| 2013-09-24
      
      § | 
    
  | 15:07 | <cmjohnson> | db1053 removing/replacing disk 3 | [production] | 
            
  | 15:05 | <yurik> | synchronized php-1.22wmf17/extensions/ZeroRatedMobileAccess/ | [production] | 
            
  | 14:59 | <cmjohnson> | db1001 removing/replacing disk 7 | [production] | 
            
  | 14:48 | <cmjohnson> | db1031 removing/replacing disk 7 | [production] | 
            
  | 14:38 | <cmjohnson> | replacing mw1046 hard drive --requires reinstall | [production] | 
            
  | 14:21 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Tue Sep 24 14:21:10 UTC 2013 | [production] | 
            
  | 14:15 | <LocalisationUpdate> | completed (1.22wmf17) at Tue Sep 24 14:15:56 UTC 2013 | [production] | 
            
  | 14:15 | <LocalisationUpdate> | completed (1.22wmf18) at Tue Sep 24 14:15:16 UTC 2013 | [production] | 
            
  | 14:13 | <demon> | synchronized php-1.22wmf18/extensions/Elastica  'Sync message fixes' | [production] | 
            
  | 12:26 | <mutante> | shutting down sq45 | [production] | 
            
  | 11:04 | <mutante> | powercycled sq45 | [production] | 
            
  | 10:08 | <p858snake|l> | [20:01AEST] <mark> upped to weight 5 | [production] | 
            
  | 10:08 | <p858snake|l> | [19:51AEST] <mark> !log Pooled Varnish Text caches in eqiad with weight 1 (affecting wikidata, wikivoyage, *.wikimedia.org) | [production] | 
            
  | 10:07 | <p858snake|l> | [19:36AEST] <akosiaris> !log rebooted labstore4, controller configuration lost ? | [production] | 
            
  | 10:07 | <p858snake|l> | [19:36AEST] <akosiaris> !log rebooted labstore3 (thought the 14days NFS issue was resolved???) | [production] | 
            
  | 10:04 | <akosiaris> | rebooted labstore4, controller configuration lost ? | [production] | 
            
  | 10:04 | <akosiaris> | rebooted labstore3 (thought the 14days NFS issue was resolved???) | [production] | 
            
  | 08:30 | <hashar> | restarting Jenkins again for one last plugin upgrade | [production] | 
            
  | 08:30 | <springle> | labstore4 disk issues, /a failed mount in console, ssh key bouncing. awaiting input from ryan or coren | [production] | 
            
  | 08:30 | <hashar> | 8:20am INFO: Jenkins is fully up and running | [production] | 
            
  | 08:20 | <hashar> | restarting Jenkins | [production] | 
            
  | 08:12 | <hashar> | jenkins: upgrading plugins | [production] | 
            
  | 08:12 | <hashar> | 8:12am INFO: Jenkins is fully up and running | [production] | 
            
  | 08:06 | <springle> | labstore4 locked up, power cycled | [production] | 
            
  | 07:43 | <hashar> | upgrading Jenkins on gallium from 1.509.2 to 1.509.3 | [production] | 
            
  | 06:36 | <springle> | upgrading db1002 to precise and mariadb | [production] | 
            
  | 06:13 | <^demon|zzz> | ES: enwikisource & cawiki done indexing. Really need to revisit batch indexing, way too slow. | [production] | 
            
  | 03:42 | <springle> | synchronized wmf-config/db-eqiad.php  'depool db1002 for upgrade, db1036 take up slack' | [production] | 
            
  | 03:17 | <springle> | synchronized wmf-config/db-eqiad.php | [production] | 
            
  | 02:40 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Tue Sep 24 02:40:38 UTC 2013 | [production] | 
            
  | 02:21 | <LocalisationUpdate> | completed (1.22wmf17) at Tue Sep 24 02:21:27 UTC 2013 | [production] | 
            
  | 02:15 | <LocalisationUpdate> | completed (1.22wmf18) at Tue Sep 24 02:15:01 UTC 2013 | [production] | 
            
  
    | 2013-09-23
      
      § | 
    
  | 23:36 | <RoanKattouw> | mw1072 is unhappy, sync-file throws warnings about read-only filesystem | [production] | 
            
  | 23:35 | <catrope> | synchronized php-1.22wmf18/extensions/VisualEditor  'VisualEditor cherry-picks' | [production] | 
            
  | 23:35 | <catrope> | synchronized php-1.22wmf17/extensions/VisualEditor  'VisualEditor cherry-picks' | [production] | 
            
  | 23:20 | <catrope> | synchronized wmf-config/InitialiseSettings.php  'Set VisualEditor to opt-in on enwiki' | [production] | 
            
  | 20:57 | <demon> | synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 20:48 | <reedy> | synchronized wmf-config/  'touch' | [production] | 
            
  | 20:46 | <reedy> | synchronized php-1.22wmf18/extensions/UploadWizard | [production] | 
            
  | 20:09 | <Reedy> | mw1072 has a r/o filesystem | [production] | 
            
  | 20:05 | <demon> | synchronized wmf-config/InitialiseSettings.php  'Cirrus as primary for itwikt, secondary on cawiki and enwikisource' | [production] | 
            
  | 19:46 | <reedy> | synchronized wmf-config  'touch' | [production] | 
            
  | 19:36 | <reedy> | synchronized database lists files: | [production] | 
            
  | 19:35 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 18:50 | <Ryan_Lane> | enabling cirrussearch for wikitech | [production] | 
            
  | 18:43 | <Reedy> | Created sites and site_identifier tables on commonswiki | [production] | 
            
  | 18:41 | <Ryan_Lane> | upgrading wikitech-static to 1.22wmf18 | [production] | 
            
  | 18:41 | <Ryan_Lane> | upgraded wikitech to 1.22wmf18 | [production] |