| 
      
        2013-02-14
      
      §
     | 
  
    
  | 15:22 | 
  <demon> | 
  synchronized wmf-config/InitialiseSettings.php  'Updating wikibase sort order' | 
  [production] | 
            
  | 15:22 | 
  <demon> | 
  synchronized php-1.21wmf9/extensions/Wikibase/client/includes/SortUtils.php  'Updating wikibase sort order' | 
  [production] | 
            
  | 14:52 | 
  <mutante> | 
  re-enabled cp1044 in pybal - mobile varnish cache servers all on precise now | 
  [production] | 
            
  | 14:39 | 
  <hashar> | 
  Jenkins: adding in several extension to Zuul configuration {{gerrit|49041}} | 
  [production] | 
            
  | 14:30 | 
  <hashar> | 
  jenkins : updated all jobs using latest version jjb-config | 
  [production] | 
            
  | 13:35 | 
  <mark> | 
  Fixed fixing | 
  [production] | 
            
  | 13:27 | 
  <mutante> | 
  disabling cp1044 in pybal, reinstalling with precise | 
  [production] | 
            
  | 13:22 | 
  <mutante> | 
  re-enabling cp1043 in pybal | 
  [production] | 
            
  | 13:11 | 
  <mark> | 
  Upgraded varnish to latest version with updated range patch on cp1021-1028 | 
  [production] | 
            
  | 12:16 | 
  <mark> | 
  Inserted varnish 3.0.3plus~rc1-wm7 into the precise-wikimedia APT repository | 
  [production] | 
            
  | 12:14 | 
  <mutante> | 
  stopping varnish backend on cp1043 and starting reinstall | 
  [production] | 
            
  | 12:10 | 
  <mutante> | 
  disabling cp1043 in pybal | 
  [production] | 
            
  | 11:31 | 
  <mutante> | 
  ms1004 - full disk, deleted large HTCPpurger.log file, fixed puppet runs | 
  [production] | 
            
  | 02:29 | 
  <LocalisationUpdate> | 
  completed (1.21wmf9) at Thu Feb 14 02:29:05 UTC 2013 | 
  [production] | 
            
  | 01:17 | 
  <andrewbogott> | 
  updated OpenStackManager to git head on virt0.  Previous state is preserved (temporarily) in the local 'premerge' branch. | 
  [production] | 
            
  | 01:13 | 
  <paravoid> | 
  removing mysql & apache from emery(!) | 
  [production] | 
            
  | 00:03 | 
  <RobH> | 
  db29 os installed, handed off to peter | 
  [production] | 
            
  
    | 
      
        2013-02-13
      
      §
     | 
  
    
  | 23:50 | 
  <RobH> | 
  db29 reinstalling | 
  [production] | 
            
  | 23:48 | 
  <paravoid> | 
  reinstalling sq48 | 
  [production] | 
            
  | 23:22 | 
  <RobH> | 
  db27 and db29 will be up and down for reinstallations | 
  [production] | 
            
  | 23:17 | 
  <sbernardin> | 
  rebooting db29 per https://rt.wikimedia.org/Ticket/Display.html?id=4528 | 
  [production] | 
            
  | 23:17 | 
  <sbernardin> | 
  rebooting db27 per https://rt.wikimedia.org/Ticket/Display.html?id=4528 | 
  [production] | 
            
  | 23:11 | 
  <RobH> | 
  db1013 online and puppet updated, needs to be pushed into some db cluster service | 
  [production] | 
            
  | 22:45 | 
  <RobH> | 
  mw27 locked up, powercycling | 
  [production] | 
            
  | 22:36 | 
  <notpeter> | 
  powercycling all eqiad jobrunners | 
  [production] | 
            
  | 22:13 | 
  <RobH> | 
  adding mw1201-1208 to eqiad apache pool | 
  [production] | 
            
  | 21:44 | 
  <Reedy> | 
  Running sync-common on mw1203 | 
  [production] | 
            
  | 21:43 | 
  <reedy> | 
  synchronized php-1.21wmf9  'resync to fix file permission errors' | 
  [production] | 
            
  | 21:32 | 
  <Reedy> | 
  Running sync-common on mw1204 | 
  [production] | 
            
  | 21:30 | 
  <Reedy> | 
  Running sync-common on mw1202 | 
  [production] | 
            
  | 21:27 | 
  <RobH> | 
  db27 rebooting for reinstallation | 
  [production] | 
            
  | 21:27 | 
  <RobH> | 
  db29 offline due to hardware issue, related ticket 4526 | 
  [production] | 
            
  | 21:19 | 
  <reedy> | 
  synchronized wmf-config/CommonSettings.php | 
  [production] | 
            
  | 21:11 | 
  <csteipp> | 
  synchronized wmf-config | 
  [production] | 
            
  | 21:09 | 
  <reedy> | 
  cleared profiling data | 
  [production] | 
            
  | 20:25 | 
  <reedy> | 
  synchronized wmf-config/CommonSettings.php  'Uncommenting enwiki from localClientDatabases' | 
  [production] | 
            
  | 19:59 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Enable Wikibase Client on enwiki' | 
  [production] | 
            
  | 19:47 | 
  <reedy> | 
  synchronized php-1.21wmf9/extensions/Wikibase/ | 
  [production] | 
            
  | 19:45 | 
  <siebrand> | 
  test for timestamp. | 
  [production] | 
            
  | 19:30 | 
  <aaron> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: Moved remaning wikis to 1.21wmf9 | 
  [production] | 
            
  | 19:21 | 
  <aaron> | 
  synchronized php-1.21wmf9/maintenance/runJobs.php  'deployed b03384f0ebd95e7f79638fb14ccd55da9c186d97' | 
  [production] | 
            
  | 19:12 | 
  <py> | 
  gracefulled all apaches | 
  [production] | 
            
  | 18:59 | 
  <cmjohnson1> | 
  mw1041 is borked and will be powered down and removed from network for h/w checks | 
  [production] | 
            
  | 18:27 | 
  <aaron> | 
  synchronized php-1.21wmf9/extensions/TranslationNotifications  'deployed d7bf60b57f472e5f75101f58505f61842848f961' | 
  [production] | 
            
  | 16:26 | 
  <Nemo_bis> | 
  Job queue weirdness and TranslationNotifications duplicates again noticed on Meta ([[bugzilla:42715|bug 42715]]) | 
  [production] | 
            
  | 13:33 | 
  <reedy> | 
  synchronized php-1.21wmf9/resources/mediawiki | 
  [production] | 
            
  | 09:17 | 
  <maxsem> | 
  synchronized php-1.21wmf9/extensions/MobileFrontend  'https://gerrit.wikimedia.org/r/#/c/48804/' | 
  [production] | 
            
  | 09:10 | 
  <maxsem> | 
  synchronized php-1.21wmf8/extensions/MobileFrontend  'https://gerrit.wikimedia.org/r/#/c/48804/' | 
  [production] | 
            
  | 08:52 | 
  <maxsem> | 
  synchronized php-1.21wmf8/extensions/MobileFrontend  'https://gerrit.wikimedia.org/r/#/c/48797/' | 
  [production] | 
            
  | 08:25 | 
  <hashar> | 
  jenkins: changing encryption key and regenerating secrets. See {{bug|44592}} | 
  [production] |