| 
      
        2014-06-23
      
      §
     | 
  
    
  | 15:11 | 
  <anomie> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: Adjust group rights on ruwiki [[gerrit:140910]] (duration: 00m 14s) | 
  [production] | 
            
  | 15:10 | 
  <anomie> | 
  Synchronized php-1.24wmf9/includes/api/ApiExpandTemplates.php: SWAT: Fix fatal in API action=expandtemplates with Scribunto [[gerrit:141416]] (duration: 00m 15s) | 
  [production] | 
            
  | 15:04 | 
  <anomie> | 
  Synchronized php-1.24wmf10/includes/api/ApiExpandTemplates.php: SWAT: Fix fatal in API action=expandtemplates with Scribunto [[gerrit:141417]] (duration: 00m 14s) | 
  [production] | 
            
  | 14:55 | 
  <andrewbogott> | 
  reenabling puppet on labstore1001, hoping it doesn't break labs | 
  [production] | 
            
  | 14:38 | 
  <hashar> | 
  Further upgraded Zuul up to upstream b8c24ce + our local hacks. Git tag is wmf-deploy-20140623-4 | 
  [production] | 
            
  | 14:14 | 
  <hashar> | 
  upgraded Zuul by one commit (that introduces swift supports though disabled it on our setup via a custom hack) | 
  [production] | 
            
  | 13:20 | 
  <paravoid> | 
  switching outbound email to polonium | 
  [production] | 
            
  | 12:17 | 
  <manybubbles> | 
  rebuilding Cirrus index on group0 wikis to pick up changes like results boosting from categories and wikitext search | 
  [production] | 
            
  | 10:37 | 
  <godog> | 
  powering down maerlant, decom-med | 
  [production] | 
            
  | 10:05 | 
  <godog> | 
  hardreset maerlant, stuck on console and no ssh | 
  [production] | 
            
  | 09:40 | 
  <paravoid> | 
  killing sodium's lighttpd compress cache | 
  [production] | 
            
  | 07:21 | 
  <_joe_> | 
  powercycled cp4018, stuck with a blank console | 
  [production] | 
            
  | 02:59 | 
  <springle> | 
  moving lighttpd compressed archives on sodium off / to regain inodes | 
  [production] | 
            
  | 02:46 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Mon Jun 23 02:45:24 UTC 2014 (duration 45m 23s) | 
  [production] | 
            
  | 02:27 | 
  <LocalisationUpdate> | 
  completed (1.24wmf10) at 2014-06-23 02:25:53+00:00 | 
  [production] | 
            
  | 02:15 | 
  <LocalisationUpdate> | 
  completed (1.24wmf9) at 2014-06-23 02:13:53+00:00 | 
  [production] | 
            
  | 00:38 | 
  <legoktm> | 
  mail is stuck, lots of mails queued in exim | 
  [production] | 
            
  
    | 
      
        2014-06-20
      
      §
     | 
  
    
  | 22:58 | 
  <demon> | 
  Synchronized wmf-config/CommonSettings.php: Load nostalgia from skins rather than extensions when it exists (duration: 00m 04s) | 
  [production] | 
            
  | 20:23 | 
  <demon> | 
  Synchronized wmf-config/CommonSettings.php: wfGetIP removal, code cleanup (duration: 00m 04s) | 
  [production] | 
            
  | 20:22 | 
  <demon> | 
  Synchronized wmf-config/throttle.php: wfGetIP removal, code cleanup (duration: 00m 05s) | 
  [production] | 
            
  | 17:11 | 
  <godog> | 
  expanded palladium's root to avoid filling up, suspected salt-master (RT #7721) | 
  [production] | 
            
  | 16:53 | 
  <bd808> | 
  Ran /usr/local/bin/sync-common on fenari to verify fix for bug 66844. It works! | 
  [production] | 
            
  | 15:16 | 
  <reedy> | 
  Synchronized wmf-config/extension-list-labs: (no message) (duration: 00m 16s) | 
  [production] | 
            
  | 11:00 | 
  <_joe_> | 
  restarted apache on palladium, passenger was dead and filling error logs | 
  [production] | 
            
  | 03:35 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Fri Jun 20 03:34:06 UTC 2014 (duration 34m 5s) | 
  [production] | 
            
  | 03:19 | 
  <LocalisationUpdate> | 
  completed (1.24wmf10) at 2014-06-20 03:18:36+00:00 | 
  [production] | 
            
  | 02:35 | 
  <LocalisationUpdate> | 
  completed (1.24wmf9) at 2014-06-20 02:34:14+00:00 | 
  [production] | 
            
  | 00:06 | 
  <MaxSem> | 
  Running clearMessageBlobs.php | 
  [production] | 
            
  
    | 
      
        2014-06-19
      
      §
     | 
  
    
  | 23:52 | 
  <MaxSem> | 
  that was a touch | 
  [production] | 
            
  | 23:51 | 
  <maxsem> | 
  Synchronized php-1.24wmf9/extensions/MultimediaViewer/: (no message) (duration: 00m 04s) | 
  [production] | 
            
  | 23:38 | 
  <maxsem> | 
  Finished scap: Mark Traceur made me do it! (duration: 15m 14s) | 
  [production] | 
            
  | 23:23 | 
  <maxsem> | 
  Started scap: Mark Traceur made me do it! | 
  [production] | 
            
  | 23:20 | 
  <maxsem> | 
  Synchronized php-1.24wmf10/extensions/MobileFrontend/: (no message) (duration: 00m 03s) | 
  [production] | 
            
  | 23:20 | 
  <maxsem> | 
  Synchronized php-1.24wmf9/extensions/MobileFrontend/: (no message) (duration: 00m 04s) | 
  [production] | 
            
  | 23:18 | 
  <maxsem> | 
  Synchronized php-1.24wmf10/extensions/CirrusSearch/: (no message) (duration: 00m 03s) | 
  [production] | 
            
  | 23:18 | 
  <maxsem> | 
  Synchronized php-1.24wmf10/extensions/VisualEditor/: (no message) (duration: 00m 04s) | 
  [production] | 
            
  | 23:16 | 
  <maxsem> | 
  Synchronized php-1.24wmf9/extensions/VisualEditor/: (no message) (duration: 00m 04s) | 
  [production] | 
            
  | 23:14 | 
  <bd808> | 
  Restarted logstash service on logstash1001 | 
  [production] | 
            
  | 23:06 | 
  <maxsem> | 
  Synchronized php-1.24wmf10/extensions/MultimediaViewer/: (no message) (duration: 00m 05s) | 
  [production] | 
            
  | 23:06 | 
  <maxsem> | 
  Synchronized php-1.24wmf9/extensions/MultimediaViewer/: (no message) (duration: 00m 05s) | 
  [production] |