| 
      
        2015-03-27
      
      §
     | 
  
    
  | 21:09 | 
  <demon> | 
  Synchronized wmf-config/logging-labs.php: shut up icinga, you're drunk (duration: 00m 07s) | 
  [production] | 
            
  | 21:02 | 
  <csteipp> | 
  redeploy security patches to wmf23 | 
  [production] | 
            
  | 20:10 | 
  <gwicke> | 
  thinning out old renders in restbase, keeping only the latest per revision; starting with group0, followed by wikipedia once done | 
  [production] | 
            
  | 16:59 | 
  <mutante> | 
  mount /mnt/data on praseodymium to fix cassandra | 
  [production] | 
            
  | 14:28 | 
  <_joe_> | 
  restarted mw1034, stuck in HPHP::StatCache::refresh | 
  [production] | 
            
  | 11:40 | 
  <godog> | 
  reboot ms-be1009, xfs stuck | 
  [production] | 
            
  | 07:15 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Fri Mar 27 07:14:37 UTC 2015 (duration 14m 36s) | 
  [production] | 
            
  | 02:46 | 
  <LocalisationUpdate> | 
  completed (1.25wmf23) at 2015-03-27 02:45:22+00:00 | 
  [production] | 
            
  | 02:43 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf23/cache/l10n: (no message) (duration: 03m 08s) | 
  [production] | 
            
  | 02:27 | 
  <LocalisationUpdate> | 
  completed (1.25wmf22) at 2015-03-27 02:26:36+00:00 | 
  [production] | 
            
  | 02:24 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf22/cache/l10n: (no message) (duration: 05m 03s) | 
  [production] | 
            
  | 01:01 | 
  <awight> | 
  updated payments from 32e860bd304763ccedc7110dee828249daa2b154 to f617326761887ed9a9100b472ea3b5736e2c10e6 | 
  [production] | 
            
  | 00:10 | 
  <gwicke> | 
  updated fstab data array name from md2 to md127 on cerium, xenon and praseodymium; naming changed after reboot; should probably use uuid instead | 
  [production] | 
            
  | 00:02 | 
  <mutante> | 
  remounted /mnt/data on xenon | 
  [production] | 
            
  
    | 
      
        2015-03-26
      
      §
     | 
  
    
  | 23:47 | 
  <ebernhardson> | 
  Synchronized php-1.25wmf22/extensions/EventLogging/: Bump EventLogging in 1.25wmf22 for SWAT (duration: 00m 07s) | 
  [production] | 
            
  | 23:44 | 
  <ebernhardson> | 
  Synchronized php-1.25wmf23/extensions/EventLogging/: Bump EventLogging in 1.25wmf23 for SWAT (duration: 00m 08s) | 
  [production] | 
            
  | 23:42 | 
  <mutante> | 
  starting ferm service on holmium | 
  [production] | 
            
  | 23:40 | 
  <ejegg> | 
  updated dash from 038bdc4c60697ac738eaeae384d91579710ff85a to 5a6b2dda71e6ce76d7bbba853acae8dc9416052c | 
  [production] | 
            
  | 23:34 | 
  <mutante> | 
  cerium, xenon, praseodymium - stuck at boot because /mnt/data not ready, skipped mounting to reboot | 
  [production] | 
            
  | 23:26 | 
  <gwicke> | 
  rebooted xenon, cerium, praseodymium to reload the firewall from scratch | 
  [production] | 
            
  | 23:23 | 
  <ebernhardson> | 
  Synchronized php-1.25wmf22/extensions/Flow: Bump flow submodule in 1.25wmf22 for swat (duration: 00m 08s) | 
  [production] | 
            
  | 23:21 | 
  <ebernhardson> | 
  Synchronized php-1.25wmf23/extensions/Flow/: Bump flow submodule for 1.25wmf23 (duration: 00m 09s) | 
  [production] | 
            
  | 23:19 | 
  <ebernhardson> | 
  Synchronized php-1.25wmf23/extensions/Flow/: Bump flow submodule for 1.25wmf23 (duration: 00m 09s) | 
  [production] | 
            
  | 22:39 | 
  <maxsem> | 
  Synchronized wmf-config/InitialiseSettings.php: Gather on test & test2 (duration: 00m 07s) | 
  [production] | 
            
  | 22:36 | 
  <maxsem> | 
  Synchronized php-1.25wmf23/extensions/Gather/: (no message) (duration: 00m 07s) | 
  [production] | 
            
  | 22:35 | 
  <maxsem> | 
  Synchronized php-1.25wmf22/extensions/Gather/: (no message) (duration: 00m 08s) | 
  [production] | 
            
  | 22:21 | 
  <maxsem> | 
  Finished scap: Enable Gather (duration: 31m 02s) | 
  [production] | 
            
  | 21:50 | 
  <maxsem> | 
  Started scap: Enable Gather | 
  [production] | 
            
  | 21:39 | 
  <MaxSem> | 
  Created Gather tables on test and test2 | 
  [production] | 
            
  | 20:15 | 
  <legoktm> | 
  set email for User:ThistleDew172@enwiki and attached to global | 
  [production] | 
            
  | 19:54 | 
  <mutante> | 
  praseodymium - fix firewalling | 
  [production] | 
            
  | 19:51 | 
  <mutante> | 
  praseodymium - log in via mgmt, run puppet to restore flushed iptables rules | 
  [production] | 
            
  | 19:34 | 
  <bd808> | 
  Updated iegreview to 7797bfc (Change email address used for sending out grants-related mails) for T92391 | 
  [production] | 
            
  | 19:33 | 
  <bd808> | 
  Applied schema changes to iegreview@m2-master.eqiad.wmnet for T92391 | 
  [production] | 
            
  | 18:48 | 
  <paravoid> | 
  replacing pfw1-codfw/pfw2/codfw | 
  [production] | 
            
  | 18:08 | 
  <kartik> | 
  Synchronized php-1.25wmf23/extensions/ContentTranslation: (no message) (duration: 00m 08s) | 
  [production] | 
            
  | 18:07 | 
  <kartik> | 
  Synchronized php-1.25wmf22/extensions/ContentTranslation: (no message) (duration: 00m 08s) | 
  [production] | 
            
  | 17:24 | 
  <kartik> | 
  Finished scap: Update ContentTranslation (duration: 33m 14s) | 
  [production] | 
            
  | 17:04 | 
  <ejegg> | 
  updated dash from 6d9acd60bb833c6dd57ab8b424afc8077b0c9f03 to 038bdc4c60697ac738eaeae384d91579710ff85a | 
  [production] | 
            
  | 16:51 | 
  <kartik> | 
  Started scap: Update ContentTranslation | 
  [production] | 
            
  | 15:13 | 
  <thcipriani> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT [[gerrit:199823]] (duration: 00m 06s) | 
  [production] | 
            
  | 09:57 | 
  <godog> | 
  restart keystone on virt1000 | 
  [production] | 
            
  | 06:55 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Thu Mar 26 06:53:58 UTC 2015 (duration 53m 57s) | 
  [production] | 
            
  | 05:19 | 
  <Krenair> | 
  Ran cleanup script for T92775 | 
  [production] | 
            
  | 02:47 | 
  <LocalisationUpdate> | 
  completed (1.25wmf23) at 2015-03-26 02:46:37+00:00 | 
  [production] | 
            
  | 02:45 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf23/cache/l10n: (no message) (duration: 02m 57s) | 
  [production] | 
            
  | 02:29 | 
  <LocalisationUpdate> | 
  completed (1.25wmf22) at 2015-03-26 02:28:47+00:00 | 
  [production] | 
            
  | 02:26 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf22/cache/l10n: (no message) (duration: 05m 03s) | 
  [production] | 
            
  | 00:57 | 
  <superm401> | 
  Done running FlowFixEditCount in production | 
  [production] | 
            
  | 00:03 | 
  <krenair> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/199692/ (duration: 00m 07s) | 
  [production] |