| 
      
        2014-03-04
      
      §
     | 
  
    
  | 23:09 | 
  <mutante|away> | 
  DNS update - removing 'zhen' | 
  [production] | 
            
  | 23:07 | 
  <bsitu> | 
  synchronized wmf-config/CommonSettings.php  'Update Flow cache key to 3.0' | 
  [production] | 
            
  | 23:06 | 
  <bsitu> | 
  updated /a/common to {{Gerrit|I566e00bcc}}: Update flow cache version number | 
  [production] | 
            
  | 22:58 | 
  <ebernhardson> | 
  synchronized php-1.23wmf15/extensions/Flow/ | 
  [production] | 
            
  | 22:55 | 
  <ebernhardson> | 
  synchronized php-1.23wmf16/extensions/Flow/ | 
  [production] | 
            
  | 22:31 | 
  <ori> | 
  GeoIP cookie enabled on text frontend varnish on cp1066. Appears to work well. If it causes issues, roll back by reverting I7e5ca8e54 and running 'service varnish-frontend restart'. | 
  [production] | 
            
  | 22:14 | 
  <ori> | 
  synchronized wmf-config/CommonSettings-labs.php  'Syncing lab config change I5ca0adf39 to prod for cluster consistency' | 
  [production] | 
            
  | 22:12 | 
  <ori> | 
  updated /a/common to {{Gerrit|I5ca0adf39}}: Labs: set $wgCentralGeoScriptURL to false for GeoIP cookie testing | 
  [production] | 
            
  | 20:45 | 
  <hashar> | 
  operations/apache-config.git now has a "betacluster" branch to host the beta cluster apache configuration files {{bug|56395}} | 
  [production] | 
            
  | 20:39 | 
  <catrope> | 
  synchronized wmf-config/CommonSettings.php  'Actually fix VE on officewiki this time' | 
  [production] | 
            
  | 20:38 | 
  <catrope> | 
  updated /a/common to {{Gerrit|I7ff564e4a}}: Really actually fix VE on private wikis | 
  [production] | 
            
  | 19:22 | 
  <bd808> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: group1 to 1.23wmf16 | 
  [production] | 
            
  | 19:09 | 
  <ottomata> | 
  depooled elastic1007 and elastic1013-1016 per request from manybubbles | 
  [production] | 
            
  | 19:02 | 
  <bd808> | 
  updated /a/common to {{Gerrit|I62b32888b}}: Create an autopatrolled group on itwikiquote | 
  [production] | 
            
  | 18:47 | 
  <ottomata> | 
  s/analytics/elastic in previous log message | 
  [production] | 
            
  | 18:46 | 
  <ottomata> | 
  bringing analytics1007 and analytics1013-1016 into elasticsearch cluster | 
  [production] | 
            
  | 17:21 | 
  <manybubbles> | 
  rebuilding the search index on commons after failing yesterday now that I've deployed a fix to the timeout issue | 
  [production] | 
            
  | 17:18 | 
  <manybubbles> | 
  synchronized php-1.23wmf16/extensions/CirrusSearch/  '3 small fixes including search timeouts' | 
  [production] | 
            
  | 16:13 | 
  <hashar> | 
  gallium : killed leftover jenkins instance {{bug|51817}} "Jenkins init script is crap" | 
  [production] | 
            
  | 16:07 | 
  <hashar> | 
  gallium has been sent to swap | 
  [production] | 
            
  | 15:41 | 
  <akosiaris> | 
  upgraded php5 packages, php5-wmerrors package and libmemcached11 on mw1017. This will make puppet and the corresponding icinga check unhappy.  | 
  [production] | 
            
  | 15:05 | 
  <hashar> | 
  reenabling puppet on gallium | 
  [production] | 
            
  | 14:53 | 
  <andrewbogott> | 
  switched wikitech to allow eqiad access, turned off pmtpa instance creation | 
  [production] | 
            
  | 14:53 | 
  <hashar> | 
  stopping puppet on gallium to play with apache configuration | 
  [production] | 
            
  | 13:52 | 
  <hoo> | 
  synchronized wmf-config/InitialiseSettings.php  '{{Gerrit|I62b3288}}' | 
  [production] | 
            
  | 11:19 | 
  <hashar> | 
  Restarting Jenkins, it is stalled .... | 
  [production] | 
            
  | 11:12 | 
  <hashar> | 
  Jenkins web service unavailable, investigating.  Builds should not be affected though since they dont use the web service (but gearman) | 
  [production] | 
            
  | 03:30 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at 2014-03-04 03:29:53+00:00 | 
  [production] | 
            
  | 02:46 | 
  <hoo> | 
  synchronized multiversion/activeMWVersions.php  'Comment only change {{Gerrit|I5e68518}}' | 
  [production] | 
            
  | 02:43 | 
  <hoo> | 
  synchronized multiversion/activeMWVersions.php  'Comment only change {{Gerrit' | 
  [production] | 
            
  | 02:43 | 
  <LocalisationUpdate> | 
  completed (1.23wmf16) at 2014-03-04 02:43:17+00:00 | 
  [production] | 
            
  | 02:28 | 
  <LocalisationUpdate> | 
  completed (1.23wmf15) at 2014-03-04 02:28:16+00:00 | 
  [production] | 
            
  | 02:25 | 
  <springle> | 
  schema change bug 31397 afl_namespace, slave by slave | 
  [production] | 
            
  | 01:32 | 
  <catrope> | 
  synchronized php-1.23wmf16/resources/oojs-ui/oojs-ui.js  'oojs-ui fixes' | 
  [production] | 
            
  | 01:10 | 
  <mutante> | 
  shutting down 'zhen' permanently | 
  [production] | 
            
  | 00:50 | 
  <mutante> | 
  zhen - disable puppet,revoke puppet cert,delete salt key,delete stored configs, disable monitoring... | 
  [production] | 
            
  | 00:25 | 
  <catrope> | 
  synchronized wmf-config/CommonSettings.php  'Fix VE on officewiki' | 
  [production] | 
            
  | 00:21 | 
  <catrope> | 
  synchronized php-1.23wmf16/extensions/VisualEditor/modules/ve-mw/ui/dialogs/ve.ui.MWMediaInsertDialog.js  'https://gerrit.wikimedia.org/r/#/c/116211/' | 
  [production] | 
            
  
    | 
      
        2014-03-03
      
      §
     | 
  
    
  | 23:16 | 
  <mutante> | 
  zhen - stopping redis-server, preparing for decom  | 
  [production] | 
            
  | 22:46 | 
  <mutante> | 
  powercycling ms-be1003 | 
  [production] | 
            
  | 22:33 | 
  <mutante> | 
  DNS update - removing yvon  | 
  [production] | 
            
  | 22:21 | 
  <mutante> | 
  yvon - deleted puppet stored configs, removed from icinga, shutdown -h now, kthxbye yvon | 
  [production] | 
            
  | 22:08 | 
  <catrope> | 
  synchronized wmf-config/CommonSettings.php  'Plumbing for $wmgUseParsoid' | 
  [production] | 
            
  | 22:07 | 
  <catrope> | 
  synchronized wmf-config/InitialiseSettings.php  'Add $wmgUseParsoid (true for all non-private wikis)' | 
  [production] | 
            
  | 21:19 | 
  <ottomata> | 
  restarting varnishkafka everywhere to pick up librdkafka 0.8.3 upgrade | 
  [production] | 
            
  | 21:07 | 
  <mutante> | 
  restarting parsoid with salt-run deploy.restart 'parsoid/deploy' '10%' | 
  [production] | 
            
  | 21:06 | 
  <gwicke> | 
  deployed Parsoid 98936e with deploy b070bcc | 
  [production] |