| 2014-12-09
      
      ยง | 
    
  | 20:33 | <reedy> | Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 06s) | [production] | 
            
  | 20:28 | <reedy> | Synchronized multiversion/: cdb bump (duration: 00m 05s) | [production] | 
            
  | 20:25 | <Reedy> | ran sync-common on mw1203 | [production] | 
            
  | 20:24 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: non wikipedias to 1.25wmf11 | [production] | 
            
  | 20:20 | <_joe_> | repooled the last api servers | [production] | 
            
  | 20:18 | <Coren> | gave a+r to /etc/ssh/ssh_known_hosts on tin and iron | [production] | 
            
  | 20:15 | <Reedy> | mw1203 seems to be down | [production] | 
            
  | 20:14 | <reedy> | Synchronized wmf-config/CommonSettings.php: touch (duration: 01m 07s) | [production] | 
            
  | 20:12 | <reedy> | Synchronized wmf-config/CommonSettings.php: touch (duration: 01m 03s) | [production] | 
            
  | 20:00 | <awight> | update crm from 77e99a530b7c3910ca521923d97830df08a4d1b1 to 94997a37a6531f2f1d5074895d5fa2da947e03f0 | [production] | 
            
  | 19:25 | <Reedy> | that is a lie, 266 hosts failed | [production] | 
            
  | 19:24 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: non wikipedias to 1.25wmf11 | [production] | 
            
  | 18:21 | <_joe_> | api 100% on HHVM now | [production] | 
            
  | 18:21 | <_joe_> | depooling mw1207-8, repooling mw1203-06 | [production] | 
            
  | 17:31 | <csteipp> | patched for T77624 | [production] | 
            
  | 17:27 | <csteipp> | Synchronized php-1.25wmf11/extensions/Listings/Listings.body.php: (no message) (duration: 00m 07s) | [production] | 
            
  | 17:07 | <godog> | powercycle ms-be1012, no console | [production] | 
            
  | 16:42 | <_joe_> | depooling mw1201-04 | [production] | 
            
  | 16:41 | <_joe_> | repooling mw1194-1200 | [production] | 
            
  | 16:34 | <anomie> | Synchronized wmf-config/CommonSettings-labs.php: Deploy some Labs-only changes so they're not showing as undeployed (duration: 00m 05s) | [production] | 
            
  | 16:31 | <anomie> | Synchronized php-1.25wmf11/extensions/Wikidata/: SWAT: Fix issue with json dump and sites caching in Wikidata [[gerrit:178533]] (duration: 00m 15s) | [production] | 
            
  | 16:29 | <anomie> | Synchronized php-1.25wmf10/includes/filerepo/file/File.php: SWAT: Fix for broken thumbnails when the file width is in $wgThumbnailBucket [[gerrit:178529]] (duration: 01m 04s) | [production] | 
            
  | 16:18 | <anomie> | Synchronized php-1.25wmf11/includes/filerepo/file/File.php: SWAT: Fix for broken thumbnails when the file width is in $wgThumbnailBucket [[gerrit:178531]] (duration: 00m 08s) | [production] | 
            
  | 15:36 | <qchris> | restarted EventLogging's m2 writer on vanadium. Events did not get written into the database. | [production] | 
            
  | 15:22 | <_joe_> | repooling mw1190-93, depooling mw1194-1200 | [production] | 
            
  | 14:59 | <_joe_> | repooled mw1147-48 | [production] | 
            
  | 14:42 | <YuviPanda> | killed wikidata dump process (/usr/local/bin/dumpwikidatajson.sh) per hoo | [production] | 
            
  | 13:59 | <_joe_> | repooling mw1140-46, depooling mw114[78], mw119[0-3] | [production] | 
            
  | 13:00 | <_joe_> | repool mw1133-39, depooling mw1140-46 | [production] | 
            
  | 12:01 | <_joe_> | repooling mw1125-1132, depooling mw1133-39 | [production] | 
            
  | 11:30 | <godog> | restarting diamond on trusty hosts via salt | [production] | 
            
  | 11:19 | <Reedy> | [10:43:54] <_joe_> !log repooling mw1120-25, depooling mw1126-32 | [production] | 
            
  | 11:15 | <springle> | repool db1010, warm up | [production] | 
            
  | 11:15 | <springle> | kicked morebots | [production] | 
            
  | 08:13 | <_joe_> | depooling mw1115-1119 from the api pool, reimaging | [production] | 
            
  | 08:06 | <_joe_> | restarting diamond on all appservers | [production] | 
            
  | 06:01 | <ori> | Synchronized php-1.25wmf11/extensions/Math/MathInputCheckTexvc.php: Fix for fatal caused by static call to MathRenderer::getError (duration: 00m 06s) | [production] | 
            
  | 06:01 | <ori> | Synchronized php-1.25wmf10/extensions/Math/MathInputCheckTexvc.php: Fix for fatal caused by static call to MathRenderer::getError (duration: 00m 06s) | [production] | 
            
  | 04:27 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Tue Dec  9 04:27:31 UTC 2014 (duration 27m 30s) | [production] | 
            
  | 04:18 | <springle> | upgrade db1010 trusty | [production] | 
            
  | 03:33 | <springle> | Synchronized wmf-config/db-eqiad.php: depool db1010 (duration: 00m 08s) | [production] | 
            
  | 02:23 | <LocalisationUpdate> | completed (1.25wmf11) at 2014-12-09 02:23:35+00:00 | [production] | 
            
  | 02:23 | <l10nupdate> | Synchronized php-1.25wmf11/cache/l10n: (no message) (duration: 00m 01s) | [production] | 
            
  | 02:17 | <LocalisationUpdate> | completed (1.25wmf10) at 2014-12-09 02:17:11+00:00 | [production] | 
            
  | 02:17 | <l10nupdate> | Synchronized php-1.25wmf10/cache/l10n: (no message) (duration: 00m 03s) | [production] | 
            
  | 01:49 | <ori> | Synchronized wmf-config/InitialiseSettings.php: Set $wgAjaxEditStash to false while API cluster is on Zend (duration: 00m 06s) | [production] | 
            
  | 01:24 | <K4-713> | updated payments to 0e92713c0d6e5 | [production] | 
            
  | 00:13 | <maxsem> | Synchronized php-1.25wmf10/extensions/Wikidata/: https://gerrit.wikimedia.org/r/#q,178374,n,z (duration: 00m 13s) | [production] | 
            
  | 00:12 | <maxsem> | Synchronized php-1.25wmf10/extensions/VisualEditor/: https://gerrit.wikimedia.org/r/#q,178371,n,z (duration: 00m 07s) | [production] | 
            
  | 00:09 | <maxsem> | Synchronized php-1.25wmf11/extensions/Wikidata/: https://gerrit.wikimedia.org/r/#q,178375,n,z (duration: 00m 13s) | [production] |