| 
      
        2013-02-08
      
      §
     | 
  
    
  | 19:33 | 
  <RobH> | 
  celsus back online | 
  [production] | 
            
  | 19:23 | 
  <RobH> | 
  celsus rebooting to fix power mgmt cpu spike issue | 
  [production] | 
            
  | 19:02 | 
  <RobH> | 
  mw1161-1164 in service as eqiad jobrunners | 
  [production] | 
            
  | 16:32 | 
  <ottomata> | 
  updated vumi packages in apt to  vumi_0.5.0~a+143-0~ppa3 and vumi-wikipedia_0.1~a+14-0~ppa3 | 
  [production] | 
            
  | 16:09 | 
  <mark> | 
  Upgraded and rebooted cr2-eqiad (both REs) | 
  [production] | 
            
  | 15:11 | 
  <mark> | 
  Set cr2-eqiad VRRP prio to 50 to transition to backup role | 
  [production] | 
            
  | 15:10 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: | 
  [production] | 
            
  | 15:08 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: srwikibooks back to 1.21wmf9 to test variant fixes | 
  [production] | 
            
  | 14:39 | 
  <hashar> | 
  gallium installed php-luasandbox manually. Gerrit change is {{gerrit|48127}}. | 
  [production] | 
            
  | 05:47 | 
  <tstarling> | 
  synchronized php-1.21wmf9/includes/context/RequestContext.php | 
  [production] | 
            
  | 02:54 | 
  <LocalisationUpdate> | 
  completed (1.21wmf8) at Fri Feb  8 02:54:49 UTC 2013 | 
  [production] | 
            
  | 02:30 | 
  <LocalisationUpdate> | 
  completed (1.21wmf9) at Fri Feb  8 02:30:42 UTC 2013 | 
  [production] | 
            
  | 02:02 | 
  <tstarling> | 
  synchronized wmf-config/InitialiseSettings.php | 
  [production] | 
            
  | 01:48 | 
  <LeslieCarr> | 
  cr2-knams is back up on the new code | 
  [production] | 
            
  | 01:36 | 
  <LeslieCarr> | 
  cr2-knams reloading after upgrade | 
  [production] | 
            
  | 01:34 | 
  <LeslieCarr> | 
  deactivating european bgp sessions - may cause some connectivity issues as connections reset | 
  [production] | 
            
  | 01:24 | 
  <LeslieCarr> | 
  disabling notifications on icinga and nagios | 
  [production] | 
            
  | 01:03 | 
  <reedy> | 
  synchronized php-1.21wmf9/languages/classes/ | 
  [production] | 
            
  | 00:25 | 
  <olivneh> | 
  synchronized php-1.21wmf8/extensions/GettingStarted | 
  [production] | 
            
  
    | 
      
        2013-02-07
      
      §
     | 
  
    
  | 23:31 | 
  <olivneh> | 
  synchronized wmf-config/InitialiseSettings.php  'Enabling GuidedTour on additional wikis' | 
  [production] | 
            
  | 23:25 | 
  <LeslieCarr> | 
  manually cleared arp and turned managmeent interface off and on of asw-c-eqiad to solve reachability issue | 
  [production] | 
            
  | 23:02 | 
  <mflaschen> | 
  Finished syncing Wikimedia installation... : Deploy GuidedTour | 
  [production] | 
            
  | 22:47 | 
  <mflaschen> | 
  Started syncing Wikimedia installation... : Deploy GuidedTour | 
  [production] | 
            
  | 22:39 | 
  <RobH> | 
  mw82 pooled into tampa apaches | 
  [production] | 
            
  | 22:04 | 
  <ottomata> | 
  pulled power and rebooted amaranth toolserver for multichill.  Its back online.  multichill will notify whomever needs to do more work. | 
  [production] | 
            
  | 21:58 | 
  <olivneh> | 
  synchronized php-1.21wmf9/extensions/RelatedArticles/  'Fixes 44761' | 
  [production] | 
            
  | 21:58 | 
  <olivneh> | 
  synchronized php-1.21wmf8/extensions/RelatedArticles/  'Fixes 44761' | 
  [production] | 
            
  | 21:50 | 
  <RobH> | 
  mw81-mw85 in pmtpa apache service | 
  [production] | 
            
  | 21:39 | 
  <cmjohnson1> | 
  performing a hard reboot on amaranth | 
  [production] | 
            
  | 21:34 | 
  <olivneh> | 
  synchronized php-1.21wmf8/extensions/E3Experiments | 
  [production] | 
            
  | 21:33 | 
  <olivneh> | 
  synchronized php-1.21wmf9/extensions/E3Experiments | 
  [production] | 
            
  | 21:12 | 
  <olivneh> | 
  synchronized php-1.21wmf8/extensions/PostEdit | 
  [production] | 
            
  | 21:11 | 
  <olivneh> | 
  synchronized php-1.21wmf9/extensions/PostEdit | 
  [production] | 
            
  | 21:10 | 
  <olivneh> | 
  synchronized php-1.21wmf8/extensions/EventLogging | 
  [production] | 
            
  | 21:09 | 
  <olivneh> | 
  synchronized php-1.21wmf9/extensions/EventLogging | 
  [production] | 
            
  | 20:29 | 
  <demon> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: Being paranoid, node lists were in flux during last sync | 
  [production] | 
            
  | 20:23 | 
  <demon> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: All ku/sr wikis back to 1.21wmf8 - lang converter breakage | 
  [production] | 
            
  | 20:20 | 
  <Nemo_bis> | 
  !Wikipedia and !Wikimedia projects are/were down, issue being investigated | 
  [production] | 
            
  | 20:14 | 
  <demon> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: kuwiktionary back to 1.21wmf8 | 
  [production] | 
            
  | 20:07 | 
  <RobH> | 
  removing srv219-224 from rendering config in pmtpa | 
  [production] | 
            
  | 20:06 | 
  <mark> | 
  Rebooting cr1-sdtpa | 
  [production] | 
            
  | 19:56 | 
  <RobH> | 
  adding mw75-mw80 into tampa rendering pool | 
  [production] | 
            
  | 18:42 | 
  <mark> | 
  Rebooted re0.cr1-eqiad | 
  [production] | 
            
  | 18:05 | 
  <mark> | 
  Rebooting re1.cr1-eqiad | 
  [production] | 
            
  | 16:38 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php | 
  [production] | 
            
  | 15:46 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php | 
  [production] | 
            
  | 15:44 | 
  <mark> | 
  Rebooting cr1-esams | 
  [production] | 
            
  | 15:40 | 
  <reedy> | 
  synchronized php-1.21wmf8/cache/interwiki.cdb  'Updating 1.21wmf8 interwiki cache' | 
  [production] | 
            
  | 15:39 | 
  <reedy> | 
  synchronized php-1.21wmf9/cache/interwiki.cdb  'Updating 1.21wmf9 interwiki cache' | 
  [production] | 
            
  | 14:54 | 
  <mutante> | 
  granting RT group 'toolserver admins' the 'modify ticket' permission in queue esams | 
  [production] |