| 2017-03-28
      
      § | 
    
  | 14:41 | <elukey@puppetmaster1001> | conftool action : set/pooled=yes; selector: name=mw2256.codfw.wmnet | [production] | 
            
  | 14:40 | <marostegui> | Convert dewiki UNIQUE keys into PK on db1091 (commonswiki) - T17441 | [production] | 
            
  | 14:38 | <ppchelko@tin> | Finished deploy [changeprop/deploy@bfbaa17]: Increase log level for processinng failures (duration: 01m 07s) | [production] | 
            
  | 14:38 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1091 - T17441 (duration: 00m 43s) | [production] | 
            
  | 14:38 | <elukey> | ran restart-hhvm on mw1242, hhvm threads stuck (dump debug in /tmp/hhvm.9008.bt.) - HHVM 3.12 | [production] | 
            
  | 14:37 | <ppchelko@tin> | Started deploy [changeprop/deploy@bfbaa17]: Increase log level for processinng failures | [production] | 
            
  | 13:54 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1092 - T17441 (duration: 00m 43s) | [production] | 
            
  | 13:44 | <elukey> | started hhvm on mw1261 (still depooled) - no hhvm process running | [production] | 
            
  | 13:29 | <RoanKattouw> | Ran initUserPreference.php -s ores-enabled -t rcenhancedfilters   and -s ores-enabled -t oresHighlight  on plwiki and ptwiki | [production] | 
            
  | 13:22 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: Enable RCFilters beta feature on plwiki and ptwiki T158336 (duration: 00m 43s) | [production] | 
            
  | 12:58 | <moritzm> | depooled mw1261 | [production] | 
            
  | 10:39 | <ema> | upgrading twisted to 16.2.0 on lvs3001 and lvs3002 (esams primaries) T160433 | [production] | 
            
  | 10:36 | <ema> | upgrading twisted to 16.2.0 on lvs3003 and lvs3004 (esams secondaries) T160433 | [production] | 
            
  | 10:27 | <marostegui> | Convert dewiki UNIQUE keys into PK on db1092 - https://phabricator.wikimedia.org/T17441 | [production] | 
            
  | 10:14 | <elukey> | Switching hue.w.o's backend (cache misc) from anaytics1027 to thorium - T159527 | [production] | 
            
  | 10:10 | <moritzm> | upgraded mw1262 to HHVM 3.18 | [production] | 
            
  | 08:48 | <marostegui> | Convert wikidatawiki UNIQUE keys into PK on db1092 - T17441 | [production] | 
            
  | 08:48 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1092 - T17441 (duration: 00m 44s) | [production] | 
            
  | 08:29 | <akosiaris> | enable IGMP snooping on all VLANs on asw2-d-eqiad. T133387 | [production] | 
            
  | 07:19 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1089 - T17441 (duration: 00m 43s) | [production] | 
            
  | 07:18 | <moritzm> | installing eject security updates on trusty hosts | [production] | 
            
  | 06:11 | <marostegui> | Keep converting unique keys into PK on db1089 - T17441 | [production] | 
            
  | 06:01 | <marostegui> | Deploy schema change on s2.enwiktionary.templatelinks - on codfw master, this will generate lag on codfw slaves (which have been silenced) - T154097 | [production] | 
            
  | 05:52 | <marostegui> | Run pt-table-checksum on es2 - T161510 | [production] | 
            
  | 02:39 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Tue Mar 28 02:39:53 UTC 2017 (duration 5m 28s) | [production] | 
            
  | 02:34 | <l10nupdate@tin> | scap sync-l10n completed (1.29.0-wmf.17) (duration: 12m 37s) | [production] | 
            
  | 00:36 | <reedy@tin> | Synchronized private: Remove mwblocker.log (duration: 00m 44s) | [production] | 
            
  | 00:34 | <reedy@tin> | Synchronized wmf-config/CommonSettings.php: Remove $wgProxyList (duration: 00m 43s) | [production] | 
            
  
    | 2017-03-27
      
      § | 
    
  | 23:06 | <ebernhardson@tin> | Synchronized php-1.29.0-wmf.17/extensions/WikimediaEvents/modules/ext.wikimediaEvents.searchSatisfaction.js: SWAT T160006, turning off cirrussearch AB test for sistersearch (duration: 00m 44s) | [production] | 
            
  | 22:00 | <bawolff> | deployed patch T151735 | [production] | 
            
  | 21:27 | <andrewbogott> | disabling puppet on labvirt* and labcontrol* to stagger roll out of https://gerrit.wikimedia.org/r/#/c/344689/ | [production] | 
            
  | 20:29 | <arlolra> | Updated Parsoid to 6eaad376 (T160599, T161178, T133267) | [production] | 
            
  | 20:21 | <arlolra@tin> | Finished deploy [parsoid/deploy@371ba4f]: Updating Parsoid to 6eaad376 (duration: 07m 06s) | [production] | 
            
  | 20:14 | <arlolra@tin> | Started deploy [parsoid/deploy@371ba4f]: Updating Parsoid to 6eaad376 | [production] | 
            
  | 19:57 | <mutante> | ruthenium/varnish misc - remove parsoid-tests.wikimedia.org server_name / backend - replaced by parsoid-rt-test and parsoid-vd-tests | [production] | 
            
  | 19:56 | <legoktm@tin> | Synchronized wmf-config/InitialiseSettings.php: Linter: whitelist parsoid canaries too - https://gerrit.wikimedia.org/r/#/c/344998/ - T160573 (duration: 00m 44s) | [production] | 
            
  | 18:08 | <mobrovac@tin> | Finished deploy [mobileapps/deploy@92f693c]: Remove the proxy from the config, deploying to scb2004 (duration: 00m 43s) | [production] | 
            
  | 18:07 | <mobrovac@tin> | Started deploy [mobileapps/deploy@92f693c]: Remove the proxy from the config, deploying to scb2004 | [production] | 
            
  | 18:05 | <mobrovac@tin> | Finished deploy [mobileapps/deploy@92f693c]: Remove the proxy from the config (duration: 03m 29s) | [production] | 
            
  | 18:02 | <mobrovac@tin> | Started deploy [mobileapps/deploy@92f693c]: Remove the proxy from the config | [production] | 
            
  | 17:19 | <mutante> | tin/mira: welcome new mediawiki deployer 'musikanimal' (T161181) | [production] | 
            
  | 17:03 | <gehel@tin> | Finished deploy [wdqs/wdqs@d07586c]: (no justification provided) (duration: 01m 26s) | [production] | 
            
  | 17:02 | <gehel@tin> | Started deploy [wdqs/wdqs@d07586c]: (no justification provided) | [production] | 
            
  | 16:40 | <mobrovac> | restbase deploying f53bec41 | [production] | 
            
  | 16:34 | <_joe_> | cleaned the bc cache on mw1261, restarted hhvm and repooled | [production] | 
            
  | 15:46 | <mobrovac@tin> | Finished deploy [mobileapps/deploy@aed916b]: Add discovery.wmnet to no_proxy_list (duration: 04m 05s) | [production] | 
            
  | 15:42 | <mobrovac@tin> | Started deploy [mobileapps/deploy@aed916b]: Add discovery.wmnet to no_proxy_list | [production] | 
            
  | 15:38 | <mobrovac@tin> | Finished deploy [cxserver/deploy@40e86ad]: Add discovery.wmnet to no_proxy_list (duration: 02m 39s) | [production] | 
            
  | 15:35 | <mobrovac@tin> | Started deploy [cxserver/deploy@40e86ad]: Add discovery.wmnet to no_proxy_list | [production] | 
            
  | 14:33 | <dcausse> | rebuilding ttmserver index in elastic@codfw from wasat | [production] |