| 
      
        2014-09-05
      
      §
     | 
  
    
  | 04:01 | 
  <springle> | 
  reboot es1002, fs check | 
  [production] | 
            
  | 03:47 | 
  <LocalisationUpdate> | 
  completed (1.24wmf20) at 2014-09-05 03:46:28+00:00 | 
  [production] | 
            
  | 03:46 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: depool es1002 for upgrade (duration: 00m 07s) | 
  [production] | 
            
  | 03:37 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: repool db1062 and db1068 (duration: 02m 06s) | 
  [production] | 
            
  | 03:10 | 
  <LocalisationUpdate> | 
  completed (1.24wmf19) at 2014-09-05 03:09:20+00:00 | 
  [production] | 
            
  | 02:56 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: depool db1062 and db1068 for upgrade (duration: 00m 56s) | 
  [production] | 
            
  | 02:39 | 
  <LocalisationUpdate> | 
  completed (1.24wmf15) at 2014-09-05 02:38:32+00:00 | 
  [production] | 
            
  | 01:01 | 
  <manybubbles> | 
  applied same elasticsearch configuration to dewiki, eswiki, zhwiki, and frwiki | 
  [production] | 
            
  | 00:18 | 
  <manybubbles> | 
  configured elasticsearch to force enwiki's content shards to stay off of the same nodes.  That ought to help performance. | 
  [production] | 
            
  
    | 
      
        2014-09-04
      
      §
     | 
  
    
  | 23:34 | 
  <catrope> | 
  Synchronized php-1.24wmf20/extensions/VisualEditor/: (no message) (duration: 00m 05s) | 
  [production] | 
            
  | 23:34 | 
  <catrope> | 
  Synchronized php-1.24wmf20/extensions/ZeroPortal/: (no message) (duration: 00m 04s) | 
  [production] | 
            
  | 23:34 | 
  <catrope> | 
  Synchronized php-1.24wmf20/extensions/Flow/: (no message) (duration: 00m 05s) | 
  [production] | 
            
  | 23:34 | 
  <catrope> | 
  Synchronized php-1.24wmf20/includes/specials/: (no message) (duration: 00m 04s) | 
  [production] | 
            
  | 23:31 | 
  <catrope> | 
  Synchronized php-1.24wmf19/extensions/ZeroPortal: (no message) (duration: 00m 05s) | 
  [production] | 
            
  | 23:30 | 
  <catrope> | 
  Synchronized php-1.24wmf19/extensions/Flow: (no message) (duration: 00m 05s) | 
  [production] | 
            
  | 22:52 | 
  <reedy> | 
  Finished scap: consistency (duration: 20m 44s) | 
  [production] | 
            
  | 22:31 | 
  <reedy> | 
  Started scap: consistency | 
  [production] | 
            
  | 22:28 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Thu Sep  4 22:27:45 UTC 2014 (duration 54m 38s) | 
  [production] | 
            
  | 21:54 | 
  <bd808> | 
  sync-dir failure was really on osmium, not mw1161; confusing error messages are confusing | 
  [production] | 
            
  | 21:50 | 
  <bd808> | 
  Running sync-common on mw1161 to try and reproduce error seen during sync-file | 
  [production] | 
            
  | 21:43 | 
  <spage> | 
  Synchronized wmf-config/InitialiseSettings.php: Enable Flow on pages, including frwiki and hewiki (duration: 00m 09s) | 
  [production] | 
            
  | 21:40 | 
  <spage> | 
  updated /a/common to {{Gerrit|Ib0aaa60f0}}: Enable Flow on several pages | 
  [production] | 
            
  | 21:08 | 
  <LocalisationUpdate> | 
  completed (1.24wmf20) at 2014-09-04 21:07:10+00:00 | 
  [production] | 
            
  | 20:56 | 
  <MaxSem> | 
  Running cleanupPageProps.php from terbium, now for realz | 
  [production] | 
            
  | 20:42 | 
  <mutante> | 
  restarting icinga-wm, making it join #wikidata for custom output | 
  [production] | 
            
  | 20:16 | 
  <LocalisationUpdate> | 
  completed (1.24wmf19) at 2014-09-04 20:15:35+00:00 | 
  [production] | 
            
  | 19:56 | 
  <Reedy> | 
  mw1088 and mw1100 rsync errors during the manual l10n update | 
  [production] | 
            
  | 19:25 | 
  <LocalisationUpdate> | 
  completed (1.24wmf15) at 2014-09-04 19:23:57+00:00 | 
  [production] | 
            
  | 18:32 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.24wmf20 | 
  [production] | 
            
  | 18:26 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedias to 1.24wmf19 | 
  [production] | 
            
  | 18:11 | 
  <reedy> | 
  Synchronized php-1.24wmf19: (no message) (duration: 00m 55s) | 
  [production] | 
            
  | 18:10 | 
  <reedy> | 
  Synchronized php-1.24wmf20: (no message) (duration: 00m 35s) | 
  [production] | 
            
  | 18:09 | 
  <reedy> | 
  Finished scap: testwiki to 1.24wmf20 and build l10n cache (duration: 41m 33s) | 
  [production] | 
            
  | 18:05 | 
  <mutante> | 
  restarting service gitblit on antimony | 
  [production] | 
            
  | 17:48 | 
  <RobH> | 
  correction, simply surpressing alerts for the host in icinga is the better move, as the host isnt reclaimed yet, so not removing holmium from pupeptstoreddb | 
  [production] | 
            
  | 17:46 | 
  <RobH> | 
  stopping puppet on holmium and removing it from puppetstoreddb so it doesnt show in icinga once updated | 
  [production] | 
            
  | 17:45 | 
  <RobH> | 
  shutting down holmium, as blog has migrated for a month now.  Not yet wiping system, please leave for me (robh) | 
  [production] | 
            
  | 17:27 | 
  <reedy> | 
  Started scap: testwiki to 1.24wmf20 and build l10n cache | 
  [production] | 
            
  | 16:44 | 
  <reedy> | 
  Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 17s) | 
  [production] | 
            
  | 16:36 | 
  <reedy> | 
  Synchronized wmf-config/: (no message) (duration: 00m 14s) | 
  [production] | 
            
  | 15:53 | 
  <andrew> | 
  Synchronized private/WikitechPrivateSettings.php: (no message) (duration: 00m 01s) | 
  [production] | 
            
  | 15:41 | 
  <reedy> | 
  Synchronized wmf-config/: (no message) (duration: 00m 14s) | 
  [production] | 
            
  | 15:18 | 
  <demon> | 
  Synchronized wmf-config/InitialiseSettings.php: plwiki gets Cirrus (duration: 00m 06s) | 
  [production] | 
            
  | 14:56 | 
  <bd808> | 
  ori updated scap to 773f95f (change deploy_dir to /srv/mediawiki) ~15 hours ago | 
  [production] | 
            
  | 08:16 | 
  <_joe_> | 
  running sync-common on mw1017, trying to debug the hhvm bad state | 
  [production] | 
            
  | 06:37 | 
  <godog> | 
  clear slowlog on elastic1004 | 
  [production] | 
            
  | 05:25 | 
  <jeremyb> | 
  temp hack fixed deployed for morebots (here and labs, not the other instances) | 
  [production] | 
            
  | 04:47 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: repool db1035, warm up (duration: 00m 08s) | 
  [production] | 
            
  | 04:32 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Thu Sep  4 04:31:28 UTC 2014 (duration 31m 27s) | 
  [production] | 
            
  | 03:43 | 
  <LocalisationUpdate> | 
  completed (1.24wmf19) at 2014-09-04 03:42:34+00:00 | 
  [production] |