| 
      
        2013-04-08
      
      §
     | 
  
    
  | 18:33 | 
  <Jeff_Green> | 
  deployed 1.22 on payments | 
  [production] | 
            
  | 18:04 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: enwiki to 1.22wmf1 | 
  [production] | 
            
  | 17:09 | 
  <asher> | 
  synchronized wmf-config/db-eqiad.php  'adding db1058 to s1 at full weight' | 
  [production] | 
            
  | 17:05 | 
  <asher> | 
  synchronized wmf-config/db-eqiad.php  'adding db1058 to s1 at a low warmup rate' | 
  [production] | 
            
  | 11:57 | 
  <^demon> | 
  gerrit back, deployed 2.6-rc0-154-gfcdb34b which contains a temporary fix for the stream-events timeout issues. See bug 46917 for info. | 
  [production] | 
            
  | 11:52 | 
  <^demon> | 
  bringing gerrit down for urgent update | 
  [production] | 
            
  | 10:30 | 
  <hashar> | 
  gerrit: created operations/debs/libvpx for j^ . Initialized pristine-tar and upstream branches using empty commits and force push. | 
  [production] | 
            
  | 07:48 | 
  <hashar> | 
  restarting Zuul for demo purposes :-) | 
  [production] | 
            
  | 04:00 | 
  <Tim> | 
  on brewster: root partition was full, removed squid access.log and store.log and started squid | 
  [production] | 
            
  | 03:55 | 
  <Tim> | 
  on all apaches: upgrading libpoppler | 
  [production] | 
            
  | 02:25 | 
  <LocalisationUpdate> | 
  completed (1.21wmf12) at Mon Apr  8 02:25:07 UTC 2013 | 
  [production] | 
            
  | 02:18 | 
  <LocalisationUpdate> | 
  completed (1.22wmf1) at Mon Apr  8 02:18:18 UTC 2013 | 
  [production] | 
            
  
    | 
      
        2013-04-07
      
      §
     | 
  
    
  | 22:46 | 
  <reedy> | 
  synchronized wmf-config/CommonSettings.php | 
  [production] | 
            
  | 22:40 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php | 
  [production] | 
            
  | 22:39 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php | 
  [production] | 
            
  | 22:31 | 
  <reedy> | 
  synchronized wmf-config/  'Kill readerfeedback config, it aint coming back' | 
  [production] | 
            
  | 22:26 | 
  <reedy> | 
  synchronized wmf-config/ | 
  [production] | 
            
  | 22:20 | 
  <reedy> | 
  synchronized php-1.22wmf1/extensions/CentralAuth | 
  [production] | 
            
  | 11:35 | 
  <^demon> | 
  restarting gerrit, stream-events seized up again. | 
  [production] | 
            
  | 07:26 | 
  <hashar> | 
  gallium: restarted Zuul just to be sure.   Gerrit on manganese is stuck again and need to be restarted {{bug|46917}} | 
  [production] | 
            
  | 02:17 | 
  <LocalisationUpdate> | 
  completed (1.21wmf12) at Sun Apr  7 02:17:09 UTC 2013 | 
  [production] | 
            
  | 02:11 | 
  <LocalisationUpdate> | 
  completed (1.22wmf1) at Sun Apr  7 02:11:21 UTC 2013 | 
  [production] | 
            
  
    | 
      
        2013-04-05
      
      §
     | 
  
    
  | 23:10 | 
  <binasher> | 
  running iwlinks index migration on s4 (commons) | 
  [production] | 
            
  | 21:26 | 
  <asher> | 
  synchronized wmf-config/db-eqiad.php  'returning db1021 to full weight, pulling db1023' | 
  [production] | 
            
  | 21:13 | 
  <asher> | 
  synchronized wmf-config/db-eqiad.php  'returning db1021 at a low weight' | 
  [production] | 
            
  | 20:05 | 
  <asher> | 
  synchronized wmf-config/db-eqiad.php  'pulling db1021 for upgrade' | 
  [production] | 
            
  | 19:53 | 
  <binasher> | 
  streaming backup of db1056 to db1050 (converting s1 snapshot slave to file_per_table) | 
  [production] | 
            
  | 18:58 | 
  <asher> | 
  synchronized wmf-config/db-eqiad.php  's1: moving watchlist to db1049, pulling db1043, adding db1056' | 
  [production] | 
            
  | 18:49 | 
  <Jeff_Green> | 
  increased jvm heap size on silicon (payments activemq) | 
  [production] | 
            
  | 18:42 | 
  <notpeter> | 
  restarting gerrit | 
  [production] | 
            
  | 18:22 | 
  <reedy> | 
  synchronized php-1.21wmf12  'Killing active users in 1.21wmf12' | 
  [production] | 
            
  | 09:38 | 
  <hashar> | 
  Ariel kindly restarted gerrit on manganese which unbroken the stream-events process in Zuul.  It again receives events. | 
  [production] | 
            
  | 09:32 | 
  <hashar> | 
  restarted Zuul | 
  [production] | 
            
  | 09:23 | 
  <Nemo_bis> | 
  varnish esams purging still broken, e.g. https://upload.wikimedia.org/wikipedia/commons/f/fe/Wikipedia-logo-v2-km.png | 
  [production] | 
            
  | 08:58 | 
  <hashar> | 
  for Jenkins / Gerrit issue see {{bug|46917}} | 
  [production] | 
            
  | 08:47 | 
  <hashar> | 
  Jenkins no more receiving ANY events from Gerrit. Effect: no jobs are triggered. | 
  [production] | 
            
  | 02:23 | 
  <LocalisationUpdate> | 
  completed (1.21wmf12) at Fri Apr  5 02:23:11 UTC 2013 | 
  [production] | 
            
  | 02:12 | 
  <LocalisationUpdate> | 
  completed (1.22wmf1) at Fri Apr  5 02:12:34 UTC 2013 | 
  [production] | 
            
  | 00:48 | 
  <csteipp> | 
  synchronized php-1.21wmf12/includes/ | 
  [production] | 
            
  | 00:46 | 
  <csteipp> | 
  synchronized php-1.22wmf1/includes/ | 
  [production] | 
            
  | 00:29 | 
  <Tim> | 
  on fenari: testing a new version of mw-update-l10n which doesn't use stdout of mergeMessagesFileList.php | 
  [production] | 
            
  | 00:20 | 
  <tstarling> | 
  synchronized wmf-config/ExtensionMessages-1.22wmf1.php | 
  [production] | 
            
  | 00:19 | 
  <bsitu> | 
  Finished syncing Wikimedia installation... : Update Echo to master | 
  [production] | 
            
  | 00:06 | 
  <Tim> | 
  all 1.22wmf1 wikis briefly showed error messages due to display_errors being enabled when mergeMessageFileList.php generated ExtensionMessages-1.22wmf1.php. Fixed temporarily. | 
  [production] |