| 2015-01-07
      
      § | 
    
  | 19:49 | <reedy> | Synchronized wmf-config/InitialiseSettings.php: Enabling error log for a few minutes (duration: 00m 15s) | [production] | 
            
  | 19:44 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.25wmf14 | [production] | 
            
  | 19:42 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedias to 1.25wmf13 | [production] | 
            
  | 19:36 | <reedy> | Finished scap: testwiki to 1.25wmf14... (duration: 28m 32s) | [production] | 
            
  | 19:08 | <reedy> | Started scap: testwiki to 1.25wmf14... | [production] | 
            
  | 18:42 | <godog> | reboot ms-be2011, megacli in a funny state and unable to bring new drive in service | [production] | 
            
  | 17:20 | <ori> | Synchronized php-1.25wmf13/extensions/EventLogging/modules/ext.eventLogging.core.js: I5470424: Correct events to send schema name (duration: 00m 05s) | [production] | 
            
  | 17:20 | <ori> | Synchronized php-1.25wmf12/extensions/EventLogging/modules/ext.eventLogging.core.js: I5470424: Correct events to send schema name (duration: 00m 06s) | [production] | 
            
  | 17:15 | <godog> | reboot ms-be2003 | [production] | 
            
  | 15:37 | <springle> | xtrabackup clone codfw slaves db2034 db2035 db2036 db2037 db2038 db2039 db2040 from other codfw slaves | [production] | 
            
  | 14:40 | <_joe_> | upgrading hhvm on testwiki | [production] | 
            
  | 14:16 | <springle> | xtrabackup clone db1027 to db1015 | [production] | 
            
  | 11:42 | <_joe_> | reimaging mw1009-mw1012 | [production] | 
            
  | 10:19 | <godog> | reboot ms-be2003, deleted LD should disappear | [production] | 
            
  | 09:47 | <hashar> | restarting Jenkins to resolve a deadlocks with the beta cluster jobs | [production] | 
            
  | 07:53 | <_joe_> | reimaging jobrunners mw1013-mw1016 (in batch of two) | [production] | 
            
  | 06:50 | <springle> | xtrabackup clone es2008 to es2010 | [production] | 
            
  | 06:50 | <springle> | xtrabackup clone es2006 to es2007 | [production] | 
            
  | 02:05 | <kaldari> | Synchronized php-1.25wmf13/extensions/WikiGrok/: Fixing campaign generation in WikiGrok (duration: 00m 05s) | [production] | 
            
  | 01:57 | <ori> | Synchronized php-1.25wmf13/extensions/EventLogging: I69c8daf: Update EventLogging for cherry-picks (duration: 00m 05s) | [production] | 
            
  | 01:57 | <ori> | Synchronized php-1.25wmf12/extensions/EventLogging: I07d9bc8: Update EventLogging for cherry-picks (duration: 00m 06s) | [production] | 
            
  | 01:33 | <springle> | xtrabackup clone es2008 to es2009 | [production] | 
            
  | 01:24 | <kaldari> | Synchronized php-1.25wmf13/extensions/MobileFrontend/: Sync MobileFrontend in 1.25wmf13 for VE fix (duration: 00m 05s) | [production] | 
            
  | 01:11 | <springle> | xtrabackup clone es2006 to es2005 | [production] | 
            
  
    | 2015-01-06
      
      § | 
    
  | 21:49 | <awight> | update crm from 80241fd2a43f03796b416d728661470f875a590a to 8ded33737337d0c842d8c194b5cb15b25fc99c2e | [production] | 
            
  | 20:24 | <reedy> | Synchronized wmf-config/: (no message) (duration: 00m 06s) | [production] | 
            
  | 20:16 | <Reedy> | Ran namespaceDupes.php on ndswiktionary | [production] | 
            
  | 20:16 | <reedy> | Synchronized wmf-config/: Config updates (duration: 00m 06s) | [production] | 
            
  | 20:00 | <reedy> | Synchronized docroot and w: update noc index (duration: 00m 06s) | [production] | 
            
  | 19:42 | <demon> | Finished scap: mwsearch is no more (duration: 35m 40s) | [production] | 
            
  | 19:06 | <demon> | Started scap: mwsearch is no more | [production] | 
            
  | 19:03 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: nooop | [production] | 
            
  | 19:01 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: Non wikipedias to 1.25wmf13 | [production] | 
            
  | 18:30 | <godog> | upload txstatsd 1.0.0-2 to trusty-wikimedia | [production] | 
            
  | 16:22 | <anomie> | Synchronized php-1.25wmf13/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.MobileViewTarget.js: SWAT: Update setupToolbar signature in mobile target [[gerrit:182993]] (duration: 00m 06s) | [production] | 
            
  | 16:01 | <anomie> | Synchronized wmf-config/flaggedrevs.php: SWAT: Add Flexion to $wgFlaggedRevsNamespaces for dewiktionary [[gerrit:180813]] (duration: 00m 08s) | [production] | 
            
  | 10:36 | <hashar> | The Zuul issues have their root cause in Gerrit which cannot open ReviewDB. Filled as https://phabricator.wikimedia.org/T85916 | [production] | 
            
  | 10:21 | <hashar> | Restarted Zuul. Gerrit transiently died out just like ~10 hours ago which locked Zuul entirely | [production] | 
            
  | 09:41 | <springle> | started an analytics ETL run on dbstore1002. to disable: set global event_scheduler=0; | [production] | 
            
  | 09:41 | <godog> | starting hhvm-profiler-to-carbon on tungsten T85641 | [production] | 
            
  | 09:16 | <mutante> | upgraded python version on zirconium | [production] | 
            
  | 09:13 | <hashar> | Synchronized wmf-config/CommonSettings-labs.php: (no message) (duration: 00m 05s) | [production] | 
            
  | 08:38 | <andrewbogott> | restarted gerrit service on ytterbium | [production] | 
            
  | 08:11 | <hashar> | Zuul stalled for some reason :( | [production] | 
            
  | 08:07 | <andrewbogott> | restarted pdns on virt1000 and labcontrol2001 to handle the change to nembus (just in case pdns is upset by change!) | [production] | 
            
  | 08:07 | <andrewbogott> | moved codfw ldap service to nembus | [production] | 
            
  | 02:03 | <aude> | Finished scap: Add Wikidata other projects message (duration: 37m 18s) | [production] | 
            
  | 01:26 | <aude> | Started scap: Add Wikidata other projects message | [production] | 
            
  | 01:23 | <aude> | Synchronized php-1.25wmf13/extensions/WikimediaMessages: Add Wikidata other projects message (duration: 00m 06s) | [production] | 
            
  | 01:16 | <aude> | Synchronized php-1.25wmf13/extensions/MobileFrontend: Fix MobileFrontend bugs (duration: 00m 06s) | [production] |