| 2013-10-22
      
      § | 
    
  | 22:51 | <K4-713> | updated payments cluster to 300fa32 | [production] | 
            
  | 22:24 | <ori> | synchronized php-1.22wmf22/resources/mediawiki/mediawiki.inspect.js  'I0e7a47b5a: mediawiki.inspect: add CSS report (4/4)' | [production] | 
            
  | 22:23 | <ori> | synchronized php-1.22wmf22/resources/mediawiki/mediawiki.js  'I0e7a47b5a: mediawiki.inspect: add CSS report (3/4)' | [production] | 
            
  | 22:23 | <ori> | synchronized php-1.22wmf21/resources/mediawiki/mediawiki.inspect.js  'I0e7a47b5a: mediawiki.inspect: add CSS report (2/4)' | [production] | 
            
  | 22:23 | <ori> | synchronized php-1.22wmf21/resources/mediawiki/mediawiki.js  'I0e7a47b5a: mediawiki.inspect: add CSS report (1/4)' | [production] | 
            
  | 22:09 | <ori> | synchronized wmf-config/CommonSettings.php  'I6f6e0ffa3: Update beta to use loginwiki for SUL (bug 55760)' | [production] | 
            
  | 22:07 | <ori-l> | during previous sync-file: could not resolve mw125; time out connecting to srv291 | [production] | 
            
  | 22:07 | <ori> | synchronized wmf-config/Bug54847.php  'Ide07296c5: Fix error message for recycled passwords (Bug 56002)' | [production] | 
            
  | 21:38 | <reedy> | synchronized php-1.22wmf22/extensions/Wikibase | [production] | 
            
  | 21:37 | <reedy> | synchronized php-1.22wmf22/extensions/RelatedSites | [production] | 
            
  | 21:08 | <Ryan_Lane> | pushed git deployment changes 91151 and 91155 to production | [production] | 
            
  | 20:30 | <bblack> | varnish: banned 'obj.http.content-length == 33518' on text varnishes everywhere (extract2.php leakage) | [production] | 
            
  | 19:54 | <Ryan_Lane> | updating git deployment system with change 90277 (0424d6f7) | [production] | 
            
  | 19:23 | <cmjohnson1> | dns update | [production] | 
            
  | 18:42 | <bsitu> | synchronized php-1.22wmf21/extensions/Echo/modules/overlay  'touch' | [production] | 
            
  | 18:35 | <bsitu> | synchronized wmf-config/InitialiseSettings.php  'touch' | [production] | 
            
  | 18:09 | <bsitu> | synchronized echowikis.dblist  'Enable Echo and Thanks on all wikis except dewiki and itwiki' | [production] | 
            
  | 18:08 | <bsitu> | synchronized wmf-config/InitialiseSettings.php  'Enable Echo and Thanks on all wikis except dewiki and itwiki' | [production] | 
            
  | 18:01 | <mutante> | restarting a few Apaches that didnt get restarted by graceful-all (hrmmpf) | [production] | 
            
  | 17:51 | <awight> | civicrm updated 41b621ab759fbafc9dcc9cf8323d1bdd973dfb72 to 6f370cccad7f963dd1cfbc2fdd9c68a5b2288836 | [production] | 
            
  | 17:23 | <mark> | Banned Content-Type == "application/x-httpd-php" in text Varnish (eqiad) | [production] | 
            
  | 17:19 | <mutante> | apache graceful-all | [production] | 
            
  | 17:06 | <mutante> | sync-apache, graceful  to unify www portal docroots | [production] | 
            
  | 16:31 | <demon> | synchronized wmf-config/InitialiseSettings.php  'Touch' | [production] | 
            
  | 16:26 | <demon> | synchronized cirrus.dblist  'foundationwiki gets cirrus' | [production] | 
            
  | 16:19 | <reedy> | synchronized docroot/wwwportal | [production] | 
            
  | 10:09 | <apergos> | powercycled sq80, hung on mgmt console | [production] | 
            
  | 09:42 | <hashar> | Zuul: no more voting on code-review label {{bug|55757}} {{gerrit|91157}}.  Will make the database happier I guess | [production] | 
            
  | 09:30 | <springle> | synchronized wmf-config/db-eqiad.php  'repool db1018' | [production] | 
            
  | 03:46 | <springle> | synchronized wmf-config/db-eqiad.php  'repool db1007, warm up db1041' | [production] | 
            
  | 03:06 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Tue Oct 22 03:06:46 UTC 2013 | [production] | 
            
  | 03:00 | <springle> | upgrading db1017 to precise | [production] | 
            
  | 02:39 | <LocalisationUpdate> | completed (1.22wmf21) at Tue Oct 22 02:39:32 UTC 2013 | [production] | 
            
  | 02:25 | <LocalisationUpdate> | completed (1.22wmf22) at Tue Oct 22 02:25:12 UTC 2013 | [production] | 
            
  | 01:41 | <springle> | start xtrabackup clone db1007 to db1041 | [production] | 
            
  | 01:26 | <springle> | start xtrabackup clone db1018 to db1034 | [production] | 
            
  | 01:07 | <springle> | upgrading db1034 & db1041 to precise | [production] | 
            
  | 00:53 | <springle> | synchronized wmf-config/db-eqiad.php  'depool db1007 while cloning' | [production] | 
            
  | 00:46 | <awight> | crm schema change: wmf_civicrm 6008->6009 | [production] | 
            
  | 00:40 | <springle> | synchronized wmf-config/db-eqiad.php  'depool db1018 while cloning' | [production] | 
            
  | 00:36 | <awight> | manual schema change on crm: update system set schema_version=6008 where name='wmf_civicrm'; | [production] | 
            
  | 00:24 | <awight> | fundraising-crm updated from 9247de6335d74d32447944eef467960edc74ae84 to 41b621ab759fbafc9dcc9cf8323d1bdd973dfb72 | [production] | 
            
  
    | 2013-10-21
      
      § | 
    
  | 23:22 | <mwalker> | synchronized php-1.22wmf21/extensions/CentralNotice/  'Reverting earlier change' | [production] | 
            
  | 23:21 | <mwalker> | synchronized php-1.22wmf22/extensions/CentralNotice/  'Reverting earlier change' | [production] | 
            
  | 23:17 | <mwalker> | CentralNotice deploy had an issue where we are now pushing ALL traffic to the mobile site for CN -- ganglia reports a spike and I'm now reverting the change | [production] | 
            
  | 23:09 | <^demon|lunch> | elastic: index created for bnwiki, running force indexing in 4 processes on terbium | [production] | 
            
  | 23:07 | <demon> | synchronized wmf-config/InitialiseSettings.php  'bnwiki gets cirrus as secondary' | [production] | 
            
  | 23:05 | <mwalker> | synchronized php-1.22wmf22/extensions/CentralNotice/  'Updating CentralNotice to master' | [production] | 
            
  | 23:05 | <mwalker> | synchronized php-1.22wmf21/extensions/CentralNotice/  'Updating CentralNotice to master' | [production] | 
            
  | 22:49 | <paravoid> | powercycling ms-be1004, locked up(?) | [production] |