| 2014-02-21
      
      ยง | 
    
  | 22:12 | <mwalker> | updated civicrm from eb3536eb32cbc7400e4e5884d56fbf104e38fc2b to 41dce289bc15ea1ca638c37b29ff2e3e709a2251 for thank you templates | [production] | 
            
  | 21:40 | <bd808> | mw1047 and mw1079 errors cleared after apache-graceful | [production] | 
            
  | 21:29 | <mutante> | graceful'ing apache on mw1047 and mw1079 by request | [production] | 
            
  | 21:26 | <bd808> | mw1047 and mw1079 throwing PHP exception that looks like APC corruption | [production] | 
            
  | 20:35 | <bd808> | Finished scap: no-diff scap; recording asciicast (duration: 03m 13s) | [production] | 
            
  | 20:31 | <bd808> | Started scap: no-diff scap; recording asciicast | [production] | 
            
  | 18:57 | <catrope> | synchronized php-1.23wmf15/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.ViewPageTarget.js  'touch' | [production] | 
            
  | 18:57 | <catrope> | synchronized php-1.23wmf15/extensions/VisualEditor/modules/ve-mw/init/ve.init.mw.Target.js  'touch' | [production] | 
            
  | 18:57 | <catrope> | synchronized php-1.23wmf14/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.ViewPageTarget.js  'touch' | [production] | 
            
  | 18:56 | <catrope> | synchronized php-1.23wmf14/extensions/VisualEditor/modules/ve-mw/init/ve.init.mw.Target.js  'touch' | [production] | 
            
  | 18:55 | <bd808> | The 4 hosts that failed scap-rebuild-cdbs were snapshot[1234]; can we pull them from mediawiki-installation dsh group? | [production] | 
            
  | 18:54 | <bd808> | Finished scap: no-diff scap to test script changes; expect l10n updates (duration: 13m 38s) | [production] | 
            
  | 18:54 | <bd808> | scap-rebuild-cdbs failed on 4 hosts | [production] | 
            
  | 18:50 | <bd808> | The 4 hosts that failed scap-1 were snapshot[1234]; all have old/bad python installs | [production] | 
            
  | 18:49 | <bd808> | scap-1 failed on 4 hosts | [production] | 
            
  | 18:41 | <bd808> | Started scap: no-diff scap to test script changes; expect l10n updates | [production] | 
            
  | 18:36 | <bd808> | Forced update of /svr/scap to 6203585 across cluster | [production] | 
            
  | 18:25 | <ottomata> | initiating kafka preferred replica election to rebalance partition leaders | [production] | 
            
  | 18:19 | <bblack> | cp1054 healthy now, rebuilding persistent cache from scratch there... | [production] | 
            
  | 15:30 | <Jeff_Green> | dist-upgrade and reboot boron | [production] | 
            
  | 13:29 | <akosiaris> | just resized 208.80.155.64/26 to 208.80.155.64/28. This is Sandbox1-b-eqiad subnet. dickson.freenode.net needs to have it's netmask changed. I will talk with coren, mutante | [production] | 
            
  | 10:05 | <ori> | updated /a/common to {{Gerrit|I10170d77c}}: Set $wmfExtendedVersionNumber = $wmfVersionNumber | [production] | 
            
  | 09:53 | <ori> | synchronized php-1.23wmf14/extensions/MultimediaViewer/resources/mmv/mmv.performance.js  'I41b6e975353: Backport fix for stats.bandwidth == Infinity' | [production] | 
            
  | 07:32 | <springle> | synchronized wmf-config/db-eqiad.php  's3 db1027 full steam' | [production] | 
            
  | 07:24 | <springle> | synchronized wmf-config/db-eqiad.php  's3 repool db1027 warm up' | [production] | 
            
  | 07:02 | <springle> | synchronized wmf-config/db-eqiad.php  's3 depool db1027 for upgrade' | [production] | 
            
  | 05:48 | <springle> | synchronized wmf-config/db-eqiad.php  's1 repool db1056' | [production] | 
            
  | 05:33 | <springle> | synchronized wmf-config/db-eqiad.php  's1 depool db1056 schema change' | [production] | 
            
  | 05:30 | <springle> | synchronized wmf-config/db-eqiad.php  's1 repool db1051' | [production] | 
            
  | 05:14 | <springle> | synchronized wmf-config/db-eqiad.php  's1 depool db1051 schema change' | [production] | 
            
  | 04:38 | <Coren> | rebooting labnet1001 -> nova-network seems quite dead | [production] | 
            
  | 03:48 | <reedy> | updated /a/common to {{Gerrit|I8252f5a09}}: Swiched from using dat to json files for wikiversions | [production] | 
            
  | 03:48 | <reedy> | synchronized docroot/noc/conf/index.php | [production] | 
            
  | 03:45 | <LocalisationUpdate> | ResourceLoader cache refresh completed at 2014-02-21 03:45:24+00:00 | [production] | 
            
  | 03:36 | <reedy> | synchronized docroot and w | [production] | 
            
  | 03:34 | <reedy> | synchronized docroot and w | [production] | 
            
  | 03:33 | <reedy> | synchronized wikiversions-labs.json | [production] | 
            
  | 03:32 | <reedy> | synchronized wikiversions.json | [production] | 
            
  | 03:31 | <reedy> | synchronized wikiversions.cdb | [production] | 
            
  | 03:29 | <reedy> | updated /a/common to {{Gerrit|Ic27cb9581}}: Fix CDB file generation | [production] | 
            
  | 03:03 | <ori> | updated /a/common to {{Gerrit|Ic804421a6}}: Change wikiversions to use json | [production] | 
            
  | 02:57 | <LocalisationUpdate> | completed (1.23wmf15) at 2014-02-21 02:57:44+00:00 | [production] | 
            
  | 02:26 | <LocalisationUpdate> | completed (1.23wmf14) at 2014-02-21 02:26:42+00:00 | [production] | 
            
  | 01:19 | <springle> | online schema change in progress gerrit 92037 user_password_expires | [production] | 
            
  | 00:25 | <ebernhardson> | synchronized php-1.23wmf15/extensions/Flow/ | [production] | 
            
  | 00:23 | <ebernhardson> | synchronized php-1.23wmf14/extensions/Flow/ | [production] | 
            
  | 00:17 | <Krinkle> | Reloading zuul to deploy I37ce89455724ed15 | [production] | 
            
  | 00:12 | <maxsem> | synchronized php-1.23wmf14/extensions/ConfirmEdit/ | [production] | 
            
  | 00:10 | <maxsem> | synchronized php-1.23wmf14/includes/api/ApiCreateAccount.php | [production] | 
            
  | 00:07 | <maxsem> | synchronized php-1.23wmf15/extensions/MobileFrontend  'https://gerrit.wikimedia.org/r/114644' | [production] |