| 
      
        2014-03-06
      
      §
     | 
  
    
  | 14:46 | 
  <reedy> | 
  synchronized database lists files: | 
  [production] | 
            
  | 14:35 | 
  <hoo> | 
  synchronized wikidataclient.dblist  'Setup test.wikidata as repo for test2 and test.wikipedia {{Gerrit|I6f4c512}}' | 
  [production] | 
            
  | 14:35 | 
  <hoo> | 
  synchronized wmf-config/  'Setup test.wikidata as repo for test2 and test.wikipedia {{Gerrit|I6f4c512}}' | 
  [production] | 
            
  | 09:43 | 
  <faidon> | 
  synchronized wmf-config/CommonSettings.php  'revert wgCentralGeoScriptURL to false' | 
  [production] | 
            
  | 09:43 | 
  <faidon> | 
  updated /a/common to {{Gerrit|Id235690c6}}: Revert "CentralNotice: set $wgCentralGeoScriptURL to false" | 
  [production] | 
            
  | 09:37 | 
  <springle> | 
  synchronized wmf-config/db-eqiad.php  's1 db1034 full steam' | 
  [production] | 
            
  | 07:44 | 
  <springle> | 
  synchronized wmf-config/db-eqiad.php  's6 db1010 full steam, s1 db1034 warm up' | 
  [production] | 
            
  | 04:20 | 
  <springle> | 
  synchronized wmf-config/db-eqiad.php  's6 repool db1010 warm up' | 
  [production] | 
            
  | 03:34 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at 2014-03-06 03:34:41+00:00 | 
  [production] | 
            
  | 03:30 | 
  <springle> | 
  s1 xtrabackup clone db1049 to db1034 | 
  [production] | 
            
  | 03:29 | 
  <springle> | 
  synchronized wmf-config/db-eqiad.php  's1 depool db1049' | 
  [production] | 
            
  | 03:20 | 
  <springle> | 
  db1034 testing /a ext4 noatime,barrier=0 | 
  [production] | 
            
  | 02:51 | 
  <LocalisationUpdate> | 
  completed (1.23wmf16) at 2014-03-06 02:50:59+00:00 | 
  [production] | 
            
  | 02:26 | 
  <LocalisationUpdate> | 
  completed (1.23wmf15) at 2014-03-06 02:26:16+00:00 | 
  [production] | 
            
  | 01:47 | 
  <springle> | 
  s6 xtrabackup clone db1022 to db1010 | 
  [production] | 
            
  | 00:20 | 
  <catrope> | 
  synchronized php-1.23wmf16/extensions/VisualEditor/modules/ve-mw/dm/nodes/ve.dm.MWBlockImageNode.js  'Fix 2-pixel image bug' | 
  [production] | 
            
  | 00:20 | 
  <catrope> | 
  updated /a/common/php-1.23wmf16 to {{Gerrit|Id675e756a}}: Update VisualEditor extension, for the right cherry-pick this time | 
  [production] | 
            
  | 00:11 | 
  <catrope> | 
  updated /a/common/php-1.23wmf16 to {{Gerrit|I706751cd2}}: Update VisualEditor to wmf16 branch for cherry-pick | 
  [production] | 
            
  | 00:04 | 
  <anomie> | 
  synchronized php-1.23wmf16/includes/htmlform/  'Backport [[gerrit:117038]] to fix regression' | 
  [production] | 
            
  | 00:03 | 
  <anomie> | 
  synchronized php-1.23wmf15/includes/htmlform/  'Backport [[gerrit:117038]] to fix regression' | 
  [production] | 
            
  | 00:03 | 
  <^d> | 
  restarted gitblit service on antimony | 
  [production] | 
            
  
    | 
      
        2014-03-05
      
      §
     | 
  
    
  | 23:21 | 
  <bd808> | 
  Finished scap: no-diff scap to test script changes (duration: 27m 25s) | 
  [production] | 
            
  | 22:53 | 
  <bd808> | 
  Started scap: no-diff scap to test script changes | 
  [production] | 
            
  | 22:22 | 
  <bd808> | 
  scap broken; working on a fix | 
  [production] | 
            
  | 22:21 | 
  <bd808> | 
  scap failed: AttributeError 'int' object has no attribute 'find' (duration: 00m 00s) | 
  [production] | 
            
  | 21:23 | 
  <mutante> | 
  killing rhodium from puppet stored configs and icinga, was already removed from site.pp in change 115638 | 
  [production] | 
            
  | 20:48 | 
  <mutante> | 
  starting NTP on analytics1004 | 
  [production] | 
            
  | 20:46 | 
  <demon> | 
  synchronized wmf-config/InitialiseSettings.php  'Wikiquote Cirrus indexes done building, Beta for all!' | 
  [production] | 
            
  | 20:10 | 
  <mutante> | 
  installing php and security upgrades on bast1001 | 
  [production] | 
            
  | 20:09 | 
  <mutante> | 
  installing security upgrades on iron | 
  [production] | 
            
  | 18:27 | 
  <ori> | 
  synchronized wmf-config/CommonSettings.php  'Ic410bd788a: CentralNotice: set $wgCentralGeoScriptURL to false' | 
  [production] | 
            
  | 18:26 | 
  <ori> | 
  updated /a/common to {{Gerrit|Ic410bd788}}: CentralNotice: set $wgCentralGeoScriptURL to false | 
  [production] | 
            
  | 18:24 | 
  <awight> | 
  updated crm from from aa3cd54e1ebf21bc7b2c34e2431a4af32dac3b32 to f06bd7920921205e97a142cec19f1096b9761768 | 
  [production] | 
            
  | 18:24 | 
  <awight> | 
  updated crm from 648337cad8d465b2e03421aac59bc1117a797fd0 to aa3cd54e1ebf21bc7b2c34e2431a4af32dac3b32 | 
  [production] | 
            
  | 18:19 | 
  <ori> | 
  Cookie-based geolocation deployed to all text varnishes | 
  [production] | 
            
  | 17:20 | 
  <demon> | 
  synchronized wmf-config/InitialiseSettings.php  'Cirrus for all the wikiquotes' | 
  [production] | 
            
  | 16:31 | 
  <cmjohnson1> | 
  carbon going down | 
  [production] | 
            
  | 16:16 | 
  <akosiaris> | 
  upgraded php5 packages, php5-wmerrors and libmemcached packages on beta in preparation for full cluster upgrade. This will make puppet unhappy. | 
  [production] | 
            
  | 16:04 | 
  <cmjohnson1> | 
  shutting down mw1165 swapping DIMM  | 
  [production] | 
            
  | 14:40 | 
  <ottomata> | 
  starting controlled shutdown (and restart) of kafka broker analytics1022 to bring in new replica.lag settings | 
  [production] | 
            
  | 14:31 | 
  <ottomata> | 
  starting controlled shutdown of kafka broker an21 to reload new replica.lag settings | 
  [production] | 
            
  | 11:12 | 
  <hashar> | 
  jenkins back | 
  [production] | 
            
  | 11:11 | 
  <ori> | 
  updated /a/common to {{Gerrit|I5277d2451}}: Update Schema:Echo revision to 7731316 | 
  [production] | 
            
  | 11:08 | 
  <hashar> | 
  restarting Jenkins which has starving threads | 
  [production] | 
            
  | 10:03 | 
  <hashar> | 
  Jenkins: gallium overloaded with a few java threads taking 100% CPU starving the box :( | 
  [production] | 
            
  | 05:21 | 
  <Ryan_Lane> | 
  finished switch from the perl git-deploy to trigger | 
  [production] | 
            
  | 04:17 | 
  <Ryan_Lane> | 
  switching out git-deploy perl frontend for trebuchet trigger | 
  [production] | 
            
  | 03:41 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at 2014-03-05 03:41:14+00:00 | 
  [production] | 
            
  | 03:37 | 
  <springle> | 
  messing with innodb compression on db1007 | 
  [production] | 
            
  | 02:54 | 
  <LocalisationUpdate> | 
  completed (1.23wmf16) at 2014-03-05 02:54:00+00:00 | 
  [production] |