| 2014-12-11
      
      § | 
    
  | 23:44 | <bd808> | restarted logstash on logstash1001; fatalmonitor report was empty since ~20:30z | [production] | 
            
  | 23:35 | <bd808> | Synchronized wmf-config: Revert Configure logging to use MWLoggerMonologSpi (Ib8ddd86) (duration: 00m 05s) | [production] | 
            
  | 23:33 | <bd808> | Synchronized wmf-config: quick revert -- Configure logging to use MWLoggerMonologSpi (I99a032f) (duration: 00m 07s) | [production] | 
            
  | 23:30 | <bd808> | Synchronized wmf-config: Configure logging to use MWLoggerMonologSpi (I99a032f) (duration: 00m 09s) | [production] | 
            
  | 23:05 | <maxsem> | Finished scap: i18n update for CentralNotice (duration: 29m 09s) | [production] | 
            
  | 22:56 | <cscott> | updated Parsoid to version d16dd2db | [production] | 
            
  | 22:50 | <cscott> | updated OCG to version bfc3812ef346c9f767135b339cedd123a1bcac98 | [production] | 
            
  | 22:45 | <hashar> | Disconnected/reconnected the Jenkins Gearman client which unstuck Zuul magically. | [production] | 
            
  | 22:42 | <hashar> | Zuul stuck | [production] | 
            
  | 22:36 | <maxsem> | Started scap: i18n update for CentralNotice | [production] | 
            
  | 21:47 | <hashar> | Jenkins re adding [https://integration.wikimedia.org/ci/computer/integration-slave1009/ integration-slave1009] to the pool of slaves | [production] | 
            
  | 21:09 | <LocalisationUpdate> | completed (1.25wmf12) at 2014-12-11 21:09:26+00:00 | [production] | 
            
  | 21:09 | <awight> | Synchronized php-1.25wmf12/cache/l10n: (no message) (duration: 00m 01s) | [production] | 
            
  | 20:58 | <LocalisationUpdate> | completed (1.25wmf11) at 2014-12-11 20:58:24+00:00 | [production] | 
            
  | 20:58 | <awight> | Synchronized php-1.25wmf11/cache/l10n: (no message) (duration: 00m 03s) | [production] | 
            
  | 20:13 | <awight> | Synchronized php-1.25wmf12/extensions/CentralNotice: IE fix for CentralNotice hide cookies (duration: 00m 06s) | [production] | 
            
  | 20:13 | <awight> | Synchronized php-1.25wmf11/extensions/CentralNotice: IE fix for CentralNotice hide cookies (duration: 00m 07s) | [production] | 
            
  | 19:30 | <chrisjohnson> | powering down tmh1002 to replace failed disk | [production] | 
            
  | 19:21 | <legoktm> | rescuing revisions on frwiki (https://phabricator.wikimedia.org/T76979) | [production] | 
            
  | 18:41 | <godog> | restart profiler-to-carbon on tungsten to pick up changes, including hhvm-profiler-to-carbon | [production] | 
            
  | 18:20 | <ori> | Synchronized php-1.25wmf12/resources/src/mediawiki.action/mediawiki.action.edit.stash.js: Ib2de3f15: Stash edit when user idles (duration: 00m 05s) | [production] | 
            
  | 18:16 | <ejegg> | updated dash from 08b078acf904d563030ff7a37b2af8df88387e29 to 6631a97e5e3e688bc0f4d2a1f6f5d97744dba0f4 | [production] | 
            
  | 17:42 | <ottomata> | starting trusty upgrade of analytics1019 | [production] | 
            
  | 17:18 | <paravoid> | restarting apache on strontium | [production] | 
            
  | 17:06 | <_joe_> | restarting HHVM on mw1237, stuck in HPHP::StatCache::refresh | [production] | 
            
  | 16:59 | <godog> | restarted gmond on ms-fe1001, all swift machines under this aggregator were showing offline | [production] | 
            
  | 16:50 | <marktraceur> | Synchronized wmf-config/: [SWAT] [config] Actually Revert 'Configure logging to use MWLoggerMonologSpi' (duration: 00m 10s) | [production] | 
            
  | 16:44 | <marktraceur> | Synchronized wmf-config/: [SWAT] [config] Revert Configure logging to use MWLoggerMonologSpi (duration: 00m 05s) | [production] | 
            
  | 16:44 | <ottomata> | starting trusty upgrade of analytics1011 | [production] | 
            
  | 16:43 | <marktraceur> | Synchronized wmf-config/: [SWAT] [config] Configure logging to use MWLoggerMonologSpi (duration: 00m 07s) | [production] | 
            
  | 16:28 | <marktraceur> | Synchronized private/PrivateSettings.php: [SWAT] [config] Add password for logstash (duration: 00m 10s) | [production] | 
            
  | 16:25 | <marktraceur> | Synchronized php-1.25wmf12/extensions/WikimediaEvents/WikimediaEvents.php: [SWAT] [wmf12] Bump sendBeacon schema revision so new URL will be generated (duration: 00m 16s) | [production] | 
            
  | 16:23 | <marktraceur> | Synchronized php-1.25wmf11/extensions/WikimediaEvents/WikimediaEvents.php: [SWAT] [wmf11] Bump sendBeacon schema revision so new URL will be generated (duration: 00m 14s) | [production] | 
            
  | 16:12 | <marktraceur> | Synchronized wmf-config/InitialiseSettings.php: [SWAT] [config] Redisable WikiGrok on enwiki (duration: 00m 05s) | [production] | 
            
  | 16:07 | <marktraceur> | Synchronized wmf-config/InitialiseSettings.php: [SWAT] [config] Reenable WikiGrok on enwiki (duration: 00m 07s) | [production] | 
            
  | 15:59 | <ottomata> | starting trusty upgrade of analytics1033 | [production] | 
            
  | 15:04 | <hashar> | @damons we love you! | [production] | 
            
  | 15:01 | <hashar> | saved Jenkins configuration via the web interface to reset the interface language from Chinese to English | [production] | 
            
  | 13:31 | <springle> | Synchronized wmf-config/db-eqiad.php: pool db1004 in s7, warm up (duration: 00m 06s) | [production] | 
            
  | 06:03 | <ori> | restarted apache on palladium and strontium | [production] | 
            
  | 04:08 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Thu Dec 11 04:08:50 UTC 2014 (duration 8m 49s) | [production] | 
            
  | 04:08 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Thu Dec 11 04:08:33 UTC 2014 (duration 30m 22s) | [production] | 
            
  | 03:34 | <ori> | Synchronized php-1.25wmf11/extensions/Math: Ic438b307a3b46: Fix for fatal caused by static call to MathRenderer::getError (duration: 00m 06s) | [production] | 
            
  | 02:57 | <Krinkle> | git-deploy: Deploying integration/mediawiki-tools-codesniffer I602cb6cfe910fc0a | [production] | 
            
  | 02:45 | <springle> | xtrabackup clone db1007 to db1004 | [production] | 
            
  | 02:12 | <LocalisationUpdate> | completed (1.25wmf12) at 2014-12-11 02:12:04+00:00 | [production] | 
            
  | 02:12 | <l10nupdate> | Synchronized php-1.25wmf12/cache/l10n: (no message) (duration: 00m 01s) | [production] | 
            
  | 02:10 | <springle> | Synchronized wmf-config/db-eqiad.php: repool db1015, warm up (duration: 00m 08s) | [production] |