| 2014-02-06
      
      § | 
    
  | 15:28 | <reedy> | updated /a/common to {{Gerrit|I1e5ea52ae}}: s2 increase db1024 load after warm up | [production] | 
            
  | 13:13 | <paravoid> | US/Canada pacific states being served by ulsfo | [production] | 
            
  | 12:43 | <paravoid> | pointing the rest of East Asia (except CN) to ulsfo | [production] | 
            
  | 11:14 | <hashar> | jenkins: added label <tt>hasTox</tt> on integration-slave01.pmtpa.wmflabs. Will let us run tox based Jenkins jobs there. | [production] | 
            
  | 11:03 | <springle> | synchronized wmf-config/db-eqiad.php  's2 increase db1024 load after warm up' | [production] | 
            
  | 07:29 | <Ryan_Lane> | redeploying parsoid/deploy on wtp* | [production] | 
            
  | 07:01 | <springle> | xtrabackup clone db1018 to db1009 (take #2) | [production] | 
            
  | 06:48 | <springle> | synchronized wmf-config/db-eqiad.php  's2 pool db1024 warm up' | [production] | 
            
  | 05:48 | <ori> | varnish on cp1054: CPU wait spiked at 05:27. dmesg|tail: XFS: possible memory allocation deadlock in kmem_alloc. not investigating further. | [production] | 
            
  | 05:45 | <ori> | finished scap: no new code. testing scap changes. (again.) (duration: 05m 00s) | [production] | 
            
  | 05:40 | <ori> | started scap: no new code. testing scap changes. (again.) | [production] | 
            
  | 05:15 | <springle> | restart labsdb1003 mariadb instances | [production] | 
            
  | 05:09 | <Ryan_Lane> | scratch that, will redploy parsoid/deploy in about an hour | [production] | 
            
  | 05:08 | <Ryan_Lane> | redeploying parsoid/deploy on wtp* | [production] | 
            
  | 04:28 | <ori> | finished scap: no new code. testing scap changes. (duration: 04m 35s) | [production] | 
            
  | 04:24 | <ori> | started scap: no new code. testing scap changes. | [production] | 
            
  | 03:22 | <LocalisationUpdate> | ResourceLoader cache refresh completed at 2014-02-06 03:22:26+00:00 | [production] | 
            
  | 03:10 | <springle> | xtrabackup clone db1018 to db1024 | [production] | 
            
  | 02:45 | <springle> | xtrabackup clone db1034 to db1009 | [production] | 
            
  | 02:44 | <LocalisationUpdate> | completed (1.23wmf11) at 2014-02-06 02:44:10+00:00 | [production] | 
            
  | 02:23 | <springle> | synchronized wmf-config/db-eqiad.php  's2 depool db1009 schema changes' | [production] | 
            
  | 02:21 | <LocalisationUpdate> | completed (1.23wmf12) at 2014-02-06 02:21:47+00:00 | [production] | 
            
  | 02:01 | <springle> | synchronized wmf-config/db-eqiad.php  's2 repool db1034 warm up' | [production] | 
            
  | 01:06 | <mutante> | apt-get remove libnet-stomp-perl on neon, i just removed that from puppet but didn't think it should stay in as an "absent" package forever | [production] | 
            
  | 01:01 | <ori> | finished scap: no-diff scap to test script changes (duration: 02m 34s) | [production] | 
            
  | 00:58 | <ori> | started scap: no-diff scap to test script changes | [production] | 
            
  | 00:44 | <ebernhardson> | finished deploying Special:Notifications fix to Echo and Flow | [production] | 
            
  | 00:44 | <ebernhardson> | synchronized php-1.23wmf11/extensions/Flow/  'Update flow for Special:Notifications fix' | [production] | 
            
  | 00:43 | <ebernhardson> | synchronized php-1.23wmf11/extensions/Echo/  'Update echo for Special:Notifications fix' | [production] | 
            
  | 00:38 | <ebernhardson> | synchronized php-1.23wmf12/extensions/Flow/  'Update flow for Special:Notifications fix' | [production] | 
            
  | 00:37 | <ebernhardson> | synchronized php-1.23wmf12/extensions/Echo/  'Update echo for Special:Notifications fix' | [production] | 
            
  | 00:04 | <ori> | finished scap: (no message) (duration: 29m 26s) | [production] | 
            
  | 00:04 | <ori> | rebuilt wikiversions.cdb and synchronized wikiversions files: | [production] | 
            
  
    | 2014-02-05
      
      § | 
    
  | 23:34 | <ori> | started scap: (no message) | [production] | 
            
  | 22:11 | <ottomata> | stopping puppet on analytics1021.  Trying to get it to catch up on replica lag | [production] | 
            
  | 20:13 | <mwalker> | updated fundraising civicrm from ec1e1fc75193ac8c0e6237fc0b80f58b3f8d55fc to 309a69545b4c89b323bffc500f71bdee82c35e42 for logging updates | [production] | 
            
  | 18:57 | <yurik> | synchronized php-1.23wmf12/extensions/ZeroRatedMobileAccess/ | [production] | 
            
  | 18:49 | <yurik> | synchronized php-1.23wmf11/extensions/ZeroRatedMobileAccess/ | [production] | 
            
  | 18:38 | <yurik> | synchronized docroot/bits/WikipediaMobileFirefoxOS | [production] | 
            
  | 14:11 | <paravoid> | ms-be1002/sdd: megacli -DiscardPreservedCache, -CfgEachDskRaid0, puppet run | [production] | 
            
  | 13:57 | <paravoid> | remove XO-Level3 avoided as-path from cr1/2-eqiad despite no ticket reply; seems to work now | [production] | 
            
  | 13:57 | <hashar> | compressing Jenkins console logs on gallium.wikimedia.org using gzip -9 | [production] | 
            
  | 08:47 | <ori> | finished scap: (no message) (duration: 03m 31s) | [production] | 
            
  | 08:47 | <ori> | rebuilt wikiversions.cdb and synchronized wikiversions files: | [production] | 
            
  | 08:44 | <ori> | started scap: (no message) | [production] | 
            
  | 08:17 | <Ryan_Lane> | restart parsoid on wtp1012 | [production] | 
            
  | 08:14 | <Ryan_Lane> | stopping parsoid on wtp1012 shortly | [production] | 
            
  | 07:53 | <ori> | rebuilt wikiversions.cdb and synchronized wikiversions files: | [production] | 
            
  | 07:51 | <ori> | rebuilt wikiversions.cdb and synchronized wikiversions files: | [production] | 
            
  | 07:15 | <ori> | rebuilt wikiversions.cdb and synchronized wikiversions files: | [production] |