| 
      
        2015-03-30
      
      §
     | 
  
    
  | 15:30 | 
  <paravoid> | 
  reimaging nescio | 
  [production] | 
            
  | 15:06 | 
  <krenair> | 
  Synchronized wmf-config/logging.php: https://gerrit.wikimedia.org/r/#/c/200286/2 - should be a no-op (duration: 00m 08s) | 
  [production] | 
            
  | 15:02 | 
  <krenair> | 
  Synchronized visualeditor-default.dblist: https://gerrit.wikimedia.org/r/#/c/196984/ - VE phase 5 (duration: 00m 07s) | 
  [production] | 
            
  | 14:04 | 
  <godog> | 
  reload apache on iodine | 
  [production] | 
            
  | 06:25 | 
  <springle> | 
  db1035 restart failed, root fs errors | 
  [production] | 
            
  | 06:07 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Mon Mar 30 06:06:26 UTC 2015 (duration 6m 25s) | 
  [production] | 
            
  | 05:53 | 
  <springle> | 
  upgrade db1035 trusty | 
  [production] | 
            
  | 05:34 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: depool db1035 (duration: 00m 07s) | 
  [production] | 
            
  | 05:21 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: repool db1044, warm up (duration: 00m 07s) | 
  [production] | 
            
  | 05:17 | 
  <springle> | 
  upgrade db1044 trusty | 
  [production] | 
            
  | 05:17 | 
  <springle> | 
  restarted production logbot | 
  [production] | 
            
  | 05:15 | 
  <YuviPanda> | 
  restarted apache on silver | 
  [production] | 
            
  | 02:43 | 
  <LocalisationUpdate> | 
  completed (1.25wmf23) at 2015-03-30 02:42:03+00:00 | 
  [production] | 
            
  | 02:40 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf23/cache/l10n: (no message) (duration: 03m 52s) | 
  [production] | 
            
  | 02:28 | 
  <LocalisationUpdate> | 
  completed (1.25wmf22) at 2015-03-30 02:27:18+00:00 | 
  [production] | 
            
  | 02:24 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf22/cache/l10n: (no message) (duration: 05m 06s) | 
  [production] | 
            
  | 01:14 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: depool db1044 (duration: 00m 09s) | 
  [production] | 
            
  
    | 
      
        2015-03-28
      
      §
     | 
  
    
  | 23:04 | 
  <hoo> | 
  Gave sysop and checkuser to Jalexander@labswiki via shell from silver after doing it via meta failed. ([[phab:T94319|T94319]]) | 
  [production] | 
            
  | 15:06 | 
  <andrewbogott> | 
  and restarted keystone on virt1000 | 
  [production] | 
            
  | 15:06 | 
  <andrewbogott> | 
  graceful’d apache2 on virt1000 | 
  [production] | 
            
  | 10:45 | 
  <godog> | 
  powercycle ms-be1009 | 
  [production] | 
            
  | 07:53 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Sat Mar 28 07:51:56 UTC 2015 (duration 51m 55s) | 
  [production] | 
            
  | 02:52 | 
  <LocalisationUpdate> | 
  completed (1.25wmf23) at 2015-03-28 02:51:45+00:00 | 
  [production] | 
            
  | 02:48 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf23/cache/l10n: (no message) (duration: 06m 50s) | 
  [production] | 
            
  | 02:27 | 
  <LocalisationUpdate> | 
  completed (1.25wmf22) at 2015-03-28 02:26:33+00:00 | 
  [production] | 
            
  | 02:23 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf22/cache/l10n: (no message) (duration: 07m 03s) | 
  [production] | 
            
  | 00:20 | 
  <hoo> | 
  Attached local accounts to "Advance", per request: enwiki, commonswiki, metawiki, nlwiktionary and nlwikinews | 
  [production] | 
            
  
    | 
      
        2015-03-27
      
      §
     | 
  
    
  | 22:17 | 
  <robh> | 
  manually restarted zotero service on sca100[1-2] | 
  [production] | 
            
  | 21:18 | 
  <demon> | 
  Synchronized wmf-config/CommonSettings-labs.php: for completeness (duration: 00m 09s) | 
  [production] | 
            
  | 21:10 | 
  <csteipp> | 
  redeploy security patches to wmf22 | 
  [production] | 
            
  | 21:09 | 
  <demon> | 
  Synchronized wmf-config/logging-labs.php: shut up icinga, you're drunk (duration: 00m 07s) | 
  [production] | 
            
  | 21:02 | 
  <csteipp> | 
  redeploy security patches to wmf23 | 
  [production] | 
            
  | 20:10 | 
  <gwicke> | 
  thinning out old renders in restbase, keeping only the latest per revision; starting with group0, followed by wikipedia once done | 
  [production] | 
            
  | 16:59 | 
  <mutante> | 
  mount /mnt/data on praseodymium to fix cassandra | 
  [production] | 
            
  | 14:28 | 
  <_joe_> | 
  restarted mw1034, stuck in HPHP::StatCache::refresh | 
  [production] | 
            
  | 11:40 | 
  <godog> | 
  reboot ms-be1009, xfs stuck | 
  [production] | 
            
  | 07:15 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Fri Mar 27 07:14:37 UTC 2015 (duration 14m 36s) | 
  [production] | 
            
  | 02:46 | 
  <LocalisationUpdate> | 
  completed (1.25wmf23) at 2015-03-27 02:45:22+00:00 | 
  [production] | 
            
  | 02:43 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf23/cache/l10n: (no message) (duration: 03m 08s) | 
  [production] | 
            
  | 02:27 | 
  <LocalisationUpdate> | 
  completed (1.25wmf22) at 2015-03-27 02:26:36+00:00 | 
  [production] | 
            
  | 02:24 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf22/cache/l10n: (no message) (duration: 05m 03s) | 
  [production] | 
            
  | 01:01 | 
  <awight> | 
  updated payments from 32e860bd304763ccedc7110dee828249daa2b154 to f617326761887ed9a9100b472ea3b5736e2c10e6 | 
  [production] | 
            
  | 00:10 | 
  <gwicke> | 
  updated fstab data array name from md2 to md127 on cerium, xenon and praseodymium; naming changed after reboot; should probably use uuid instead | 
  [production] | 
            
  | 00:02 | 
  <mutante> | 
  remounted /mnt/data on xenon | 
  [production] |