| 2014-11-05
      
      § | 
    
  | 06:29 | <akosiaris> | depool wtp1013, wtp1014, wtp1015, wtp1016, wtp1023 for trusty reinstallation | [production] | 
            
  | 05:53 | <ori> | ran: salt -G php:hhvm cmd.run 'restart hhvm' | [production] | 
            
  | 05:26 | <ori> | Synchronized wmf-config/CommonSettings.php: $wgPercentHHVM: back to 5% (duration: 00m 11s) | [production] | 
            
  | 03:21 | <^d|voted> | restarted lucene-search-2 on search1019: it'd been timing out for a few days and filled disk with log files. | [production] | 
            
  | 02:25 | <ori> | Synchronized wmf-config/CommonSettings.php: If866e9caf: $wgPercentHHVM: 5 => 10, to test https://phabricator.wikimedia.org/T820#18870 (duration: 00m 04s) | [production] | 
            
  | 01:21 | <ori> | Synchronized php-1.25wmf5/extensions/MobileFrontend: Ic82ba72b98: Update MobileFrontend for cherry-picks (duration: 00m 04s) | [production] | 
            
  | 01:21 | <ori> | Synchronized php-1.25wmf6/extensions/MobileFrontend: Ic26f56c0d: Update MobileFrontend for cherry-picks (duration: 00m 05s) | [production] | 
            
  | 01:12 | <^d|voted> | elastic1022: banned from allocation since its unreachable. just in case it starts flapping. | [production] | 
            
  | 01:11 | <mutante> | elatic1022 - eth0: <NO-CARRIER | [production] | 
            
  | 01:07 | <ori> | upgrading HHVM app servers to 3.3.0+dfsg1-1+wm2 | [production] | 
            
  | 01:02 | <mutante> | powercycling elastic1022 | [production] | 
            
  | 00:45 | <^d|voted> | elasticsearch: rebuilding all cirrus indexes for all wikis from a screen on terbium, going to take awhile. should be boring, but if causing problems kill it first and then find me. | [production] | 
            
  | 00:24 | <demon> | Synchronized php-1.25wmf6/includes/parser/Parser.php: (no message) (duration: 00m 04s) | [production] | 
            
  | 00:23 | <demon> | Synchronized php-1.25wmf6/includes/parser/CoreTagHooks.php: (no message) (duration: 00m 04s) | [production] | 
            
  | 00:23 | <demon> | Synchronized php-1.25wmf5/includes/parser/Parser.php: (no message) (duration: 00m 04s) | [production] | 
            
  | 00:23 | <demon> | Synchronized php-1.25wmf5/includes/parser/CoreTagHooks.php: (no message) (duration: 00m 05s) | [production] | 
            
  | 00:22 | <demon> | Synchronized php-1.25wmf6/extensions/CirrusSearch/maintenance/updateOneSearchIndexConfig.php: (no message) (duration: 00m 04s) | [production] | 
            
  | 00:22 | <demon> | Synchronized php-1.25wmf5/extensions/CirrusSearch/maintenance/updateOneSearchIndexConfig.php: (no message) (duration: 00m 04s) | [production] | 
            
  | 00:04 | <demon> | Synchronized wmf-config/CirrusSearch-common.php: (no message) (duration: 00m 04s) | [production] | 
            
  | 00:03 | <demon> | Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 07s) | [production] | 
            
  
    | 2014-11-04
      
      § | 
    
  | 19:22 | <cmjohnson> | rebooting wtp1023 | [production] | 
            
  | 17:21 | <ejegg> | updated crm from b8a1fa98b5d9252d708090c99b61fd22ebe8d2be to e9e81a828d50e8bddf98eae699c925e09b25927b | [production] | 
            
  | 16:53 | <akosiaris> | repool wtp1017,wtp1018,wtp1019,wtp1020 | [production] | 
            
  | 16:50 | <hashar> | restarting Zuul/Jenkins entirely | [production] | 
            
  | 16:45 | <manybubbles> | Synchronized php-1.25wmf6/extensions/UniversalLanguageSelector/: SWAT update uls (duration: 00m 04s) | [production] | 
            
  | 16:43 | <manybubbles> | Synchronized php-1.25wmf5/extensions/UniversalLanguageSelector/: SWAT update uls (duration: 00m 04s) | [production] | 
            
  | 16:33 | <hashar> | Shutting down Jenkins to remove a deadlock :-( | [production] | 
            
  | 16:27 | <hashar> | Jenkins restarting Gearman client | [production] | 
            
  | 16:24 | <hashar> | Zuul on hold, waiting for beta cluster related jobs  to complete | [production] | 
            
  | 16:21 | <hashar> | Jenkins: disconnecting/reconnecting gearman client  , killing deployment-bastion.eqiad slave in an attempt to remove a deadlock {{bug|70597}} | [production] | 
            
  | 14:06 | <akosiaris> | upgrading kernels on amssq* | [production] | 
            
  | 13:59 | <reedy> | Synchronized wmf-config/: (no message) (duration: 00m 14s) | [production] | 
            
  | 13:58 | <Reedy> | graceful apache on mw1193 | [production] | 
            
  | 13:57 | <Reedy> | graceful apache on mw1144 | [production] | 
            
  | 13:51 | <reedy> | Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 15s) | [production] | 
            
  | 13:40 | <akosiaris> | depool wtp1017, wtp1018, wtp1019, wtp1020 from trusty reinstall | [production] | 
            
  | 13:39 | <akosiaris> | upgrading apache2 throught the mw cluster | [production] | 
            
  | 13:24 | <reedy> | Synchronized wmf-config/: (no message) (duration: 00m 19s) | [production] | 
            
  | 13:14 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: Non wikipedias to 1.25wmf6 | [production] | 
            
  | 13:03 | <reedy> | Purged l10n cache for 1.25wmf4 | [production] | 
            
  | 12:41 | <akosiaris> | repooled wtp1021,wtp1022,wtp1023 | [production] | 
            
  | 10:42 | <akosiaris> | depooled wtp1021,wtp1022,wtp1023 for re-installation with trusty | [production] | 
            
  | 06:31 | <springle> | force logrotate ocg1001 | [production] | 
            
  | 03:32 | <springle> | restart db2017 | [production] | 
            
  | 00:12 | <catrope> | Synchronized php-1.25wmf6/extensions/MultimediaViewer: SWAT (duration: 00m 03s) | [production] | 
            
  | 00:12 | <catrope> | Synchronized php-1.25wmf6/extensions/MobileFrontend: SWAT (duration: 00m 04s) | [production] | 
            
  | 00:12 | <catrope> | Synchronized php-1.25wmf6/extensions/VisualEditor: SWAT (duration: 00m 04s) | [production] | 
            
  | 00:09 | <catrope> | Synchronized php-1.25wmf5/extensions/VisualEditor: SWAT (duration: 00m 04s) | [production] | 
            
  | 00:05 | <catrope> | Synchronized wmf-config/InitialiseSettings.php: Flow on officewiki and mw.org research page (duration: 00m 04s) | [production] |