| 2015-11-10
      
      § | 
    
  | 16:44 | <krenair@tin> | Synchronized wmf-config/Wikibase.php: https://gerrit.wikimedia.org/r/#/c/252204/ (duration: 00m 34s) | [production] | 
            
  | 16:41 | <krenair@tin> | Synchronized wmf-config/Wikibase.php: https://gerrit.wikimedia.org/r/#/c/250000/ (duration: 00m 34s) | [production] | 
            
  | 16:34 | <krenair@tin> | Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/250471/ (duration: 00m 34s) | [production] | 
            
  | 16:28 | <krenair@tin> | Synchronized php-1.27.0-wmf.5/extensions/ContentTranslation/api/ApiQueryContentTranslationSuggestions.php: https://gerrit.wikimedia.org/r/#/c/252172/ (duration: 00m 35s) | [production] | 
            
  | 16:22 | <krenair@tin> | Synchronized php-1.27.0-wmf.5/extensions/Wikidata/Wikidata.php: https://gerrit.wikimedia.org/r/#/c/252227/ (duration: 00m 45s) | [production] | 
            
  | 15:51 | <mutante> | restarted krypton | [production] | 
            
  | 15:32 | <godog> | upgrade rsync on ms-be1* per T93587 | [production] | 
            
  | 14:02 | <godog> | swift codfw-prod: ms-be2017 / ms-be2019 / ms-be2021 weight 3000 | [production] | 
            
  | 10:00 | <jynus> | performing mysql maintenance of db1035 (depooled from production). Replication delay and reboots are expected. | [production] | 
            
  | 09:46 | <godog> | swift codfw-prod: ms-be2017 / ms-be2019 / ms-be2021 weight 2000 | [production] | 
            
  | 09:39 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1035 for maintenance (duration: 00m 36s) | [production] | 
            
  | 03:12 | <ottomata> | restarted statsv on hafnium just in case it stopped after kafka1013 broker crash and restart (ping ori :) ) | [production] | 
            
  | 03:10 | <ottomata> | kafka preferred-replica-election to bring kafka1013  back as leader for its partitions | [production] | 
            
  | 03:00 | <mutante> | started kafka on kafka1013 | [production] | 
            
  | 02:55 | <mutante> | tried to start kafka on kafka1018 | [production] | 
            
  | 02:31 | <mutante> | switched people.wm.org to new backend. please use "rutherfordium.eqiad.wmnet" instead of terbium now. all files have been rsynced already | [production] | 
            
  | 02:22 | <l10nupdate@tin> | Synchronized php-1.27.0-wmf.5/cache/l10n: l10nupdate for 1.27.0-wmf.5 (duration: 06m 49s) | [production] | 
            
  | 00:13 | <hoo> | Attached Richardelainechambers@commonswiki to the global account of the same name | [production] | 
            
  | 00:06 | <bd808@tin> | Synchronized wmf-config/logging.php: Disable microsecond timestamps on all loggers (bf61bfc) (duration: 00m 34s) | [production] | 
            
  
    | 2015-11-09
      
      § | 
    
  | 23:55 | <ebernhardson@tin> | Synchronized wmf-config/InitialiseSettings.php: Disable cirrus writes to labsearch, the machine cant take the load and some jobs are timing out (duration: 00m 34s) | [production] | 
            
  | 23:13 | <ebernhardson@tin> | Synchronized wmf-config/InitialiseSettings.php: Enable CirrusSearch writes to labsearch for all but enwiki and dewiki (duration: 00m 35s) | [production] | 
            
  | 22:49 | <ottomata> | rebooting analytics1032: https://phabricator.wikimedia.org/T118175 | [production] | 
            
  | 21:52 | <bearND> | MobileApps deployed sha1 6c63984 | [production] | 
            
  | 19:40 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Pool db1060 after performance patch (duration: 00m 35s) | [production] | 
            
  | 18:42 | <jynus> | restarting mysql in db1060 to test new performance configuration | [production] | 
            
  | 17:55 | <bblack> | upgrading pybal (1.10 -> 1.12) on lvs200[456].codfw.wmnet | [production] | 
            
  | 16:50 | <krenair@tin> | Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/250454/ (duration: 00m 34s) | [production] | 
            
  | 16:44 | <krenair@tin> | Synchronized php-1.27.0-wmf.5/extensions/VisualEditor/modules/ve-mw/init: https://gerrit.wikimedia.org/r/#/c/251972/ (duration: 00m 34s) | [production] | 
            
  | 16:40 | <krenair@tin> | Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/251674/ (duration: 00m 35s) | [production] | 
            
  | 16:35 | <krenair@tin> | Synchronized docroot/wikipedia.org/apple-app-site-association: https://gerrit.wikimedia.org/r/#/c/250897/ (duration: 00m 34s) | [production] | 
            
  | 16:30 | <matt_flaschen> | Ran mwscript extensions/Flow/maintenance/FlowUpdateBetaFeaturePreference.php --wiki=wikidatawiki | [production] | 
            
  | 16:24 | <krenair@tin> | Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/251677/ (duration: 00m 35s) | [production] | 
            
  | 16:14 | <krenair@tin> | Synchronized php-1.27.0-wmf.5/extensions/Flow/modules/mw.flow.Initializer.js: https://gerrit.wikimedia.org/r/#/c/251560/ (duration: 00m 44s) | [production] | 
            
  | 16:10 | <_joe_> | removing old builds from copper | [production] | 
            
  | 15:45 | <godog> | reimage graphite1002 | [production] | 
            
  | 15:06 | <ottomata> | running kafka preferred-replica-election | [production] | 
            
  | 15:02 | <ottomata> | starting kafka broker on kafka1014 | [production] | 
            
  | 14:56 | <ottomata> | stopping kafka broker on kafka1014 | [production] | 
            
  | 14:48 | <godog> | swift codfw-prod: ms-be2017 / ms-be2019 / ms-be2021 weight 1000 | [production] | 
            
  | 14:40 | <ottomata> | restarting eventlogging to see if it is ok after enabling firewall rules on kafka1014 | [production] | 
            
  | 14:36 | <_joe_> | reducing /tmp size on copper by shrinking the logical volume | [production] | 
            
  | 13:16 | <godog> | nodetool decomission restbase2001 | [production] | 
            
  | 13:13 | <mobrovac> | restbase finished deploying ae2a44f | [production] | 
            
  | 12:59 | <mobrovac> | restbase start deploying ae2a44f | [production] | 
            
  | 12:56 | <hashar> | restarting Jenkins to refresh the cli-shutdown.groovy script -- https://gerrit.wikimedia.org/r/251935 (https://phabricator.wikimedia.org/T118064) | [production] | 
            
  | 12:10 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Depooling db1060 again (duration: 01m 13s) | [production] | 
            
  | 11:16 | <hashar> | Jenkins back and apparently happy | [production] | 
            
  | 11:09 | <hashar> | Upgrading Jenkins from LTS 1.609.3 to LTS 1.625.1 https://phabricator.wikimedia.org/T118157 | [production] | 
            
  | 10:53 | <dcausse> | resuming writes to elasticsearch indices in eqiad (test) | [production] | 
            
  | 10:02 | <_joe_> | repooled mw1061 | [production] |