| 2013-04-09
      
      § | 
    
  | 23:19 | <notpeter> | running upgrades on db64 and rebooting | [production] | 
            
  | 22:55 | <maxsem> | synchronized php-1.21wmf12/extensions/MobileFrontend/ | [production] | 
            
  | 22:52 | <maxsem> | synchronized php-1.22wmf1/extensions/MobileFrontend/ | [production] | 
            
  | 22:36 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 22:33 | <LeslieCarr> | clearing a few varnish caches - fyi if db load is seen to increase | [production] | 
            
  | 22:16 | <maxsem> | Finished syncing Wikimedia installation... : Weekly mobile deployment | [production] | 
            
  | 21:45 | <maxsem> | Started syncing Wikimedia installation... : Weekly mobile deployment | [production] | 
            
  | 21:44 | <hashar> | Zuul is slowly dequeuing a large number of jobs. There is currently rougly 30 minutes lag between change submission and actually processing of the job.   No idea why but investigating. | [production] | 
            
  | 19:56 | <Ryan_Lane> | installed Nuke on wikitech | [production] | 
            
  | 18:11 | <K4-713> | updated payments anti-fraud rules | [production] | 
            
  | 13:07 | <hashar> | jenkins plugin upgrade is tracked in {{bug|47040}}. Luckily git plugin has been fixed just in time ;-D  Just need to migrate the config history. | [production] | 
            
  | 12:37 | <hashar> | jenkins plugins seems to have been upgraded overnight :( | [production] | 
            
  | 10:26 | <mark> | Deployed experimental varnish build (3.0.3plus~rc1-wm10) on dysprosium and cp1022 | [production] | 
            
  | 08:53 | <hashar> | Jenkins tests for mediawiki/core are broken ({{bug|47031}}) due to a faulty change that has been merged in master branch. Root cause is the gating job not testing the proper change {{bug| 46723}} :/ | [production] | 
            
  | 08:43 | <Nikerabbit> | Got rid of local changes and undeployed commit related to noc | [production] | 
            
  | 08:41 | <nikerabbit> | synchronized wmf-config/InitialiseSettings.php  'Narayam and WebFonts' | [production] | 
            
  | 05:41 | <Tim> | on mw1149: enabled apache access log | [production] | 
            
  | 02:31 | <LocalisationUpdate> | completed (1.21wmf12) at Tue Apr  9 02:30:59 UTC 2013 | [production] | 
            
  | 02:19 | <LocalisationUpdate> | completed (1.22wmf1) at Tue Apr  9 02:18:53 UTC 2013 | [production] | 
            
  | 00:26 | <^demon> | jenkins is back. java gc took awhile. | [production] | 
            
  | 00:19 | <^demon> | jenkins is hung, looking into it. | [production] | 
            
  | 00:16 | <awight> | payments cluster updated from 22943dc28 to d3954768e4dd | [production] | 
            
  
    | 2013-04-08
      
      § | 
    
  | 23:15 | <catrope> | synchronized php-1.22wmf1/extensions/WikiEditor  'Deploying 17a90bc74e87341199a50ebe5348576e05be028a' | [production] | 
            
  | 23:15 | <catrope> | synchronized php-1.21wmf12/extensions/WikiEditor  'Deploying 17a90bc74e87341199a50ebe5348576e05be028a' | [production] | 
            
  | 22:55 | <olivneh> | synchronized wmf-config/CommonSettings.php  'Changing  from emery to vanadium' | [production] | 
            
  | 22:13 | <demon> | synchronized php-1.21wmf12/includes/AutoLoader.php  'Backported Poolcounter convenience class' | [production] | 
            
  | 22:12 | <demon> | synchronized php-1.21wmf12/includes/PoolCounter.php  'Backported Poolcounter convenience class' | [production] | 
            
  | 21:59 | <demon> | synchronized wmf-config/PoolCounterSettings-pmtpa.php  'Enable poolcounter for search' | [production] | 
            
  | 21:58 | <demon> | synchronized wmf-config/PoolCounterSettings-eqiad.php  'Enable poolcounter for search' | [production] | 
            
  | 21:42 | <demon> | synchronized php-1.21wmf12/extensions/MWSearch/MWSearch_body.php  'Deploying Poolcounter updates' | [production] | 
            
  | 21:17 | <notpeter> | temp stopping slave on db55 | [production] | 
            
  | 20:53 | <aaron> | synchronized wmf-config/filebackend.php  'Bumped file journal ttl to a year' | [production] | 
            
  | 20:18 | <AaronSchulz> | Moved ceph sync scripts to terbium and started a second pass | [production] | 
            
  | 20:15 | <aaron> | synchronized php-1.22wmf1/extensions/FlaggedRevs  'deployed c61688baef682b7d97068fefdcc399256786a387' | [production] | 
            
  | 19:07 | <^demon> | gerrit back | [production] | 
            
  | 19:07 | <^demon> | gerrit restarting, config change to fix idle timeouts. | [production] | 
            
  | 19:00 | <ottomata> | removing some unused filters from emery's udp2log instance | [production] | 
            
  | 18:33 | <Jeff_Green> | deployed 1.22 on payments | [production] | 
            
  | 18:04 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: enwiki to 1.22wmf1 | [production] | 
            
  | 17:09 | <asher> | synchronized wmf-config/db-eqiad.php  'adding db1058 to s1 at full weight' | [production] | 
            
  | 17:05 | <asher> | synchronized wmf-config/db-eqiad.php  'adding db1058 to s1 at a low warmup rate' | [production] | 
            
  | 11:57 | <^demon> | gerrit back, deployed 2.6-rc0-154-gfcdb34b which contains a temporary fix for the stream-events timeout issues. See bug 46917 for info. | [production] | 
            
  | 11:52 | <^demon> | bringing gerrit down for urgent update | [production] | 
            
  | 10:30 | <hashar> | gerrit: created operations/debs/libvpx for j^ . Initialized pristine-tar and upstream branches using empty commits and force push. | [production] | 
            
  | 07:48 | <hashar> | restarting Zuul for demo purposes :-) | [production] | 
            
  | 04:00 | <Tim> | on brewster: root partition was full, removed squid access.log and store.log and started squid | [production] | 
            
  | 03:55 | <Tim> | on all apaches: upgrading libpoppler | [production] | 
            
  | 02:25 | <LocalisationUpdate> | completed (1.21wmf12) at Mon Apr  8 02:25:07 UTC 2013 | [production] | 
            
  | 02:18 | <LocalisationUpdate> | completed (1.22wmf1) at Mon Apr  8 02:18:18 UTC 2013 | [production] |