| 2017-06-13
      
      ยง | 
    
  | 21:56 | <hashar> | Zuul back, running in an interactive terminal. | [production] | 
            
  | 21:46 | <mutante> | netmon1002 - was able to "keyholder arm" after stretch install after applying https://gerrit.wikimedia.org/r/358884 as hotfix | [production] | 
            
  | 21:30 | <mobrovac@tin> | Finished deploy [restbase/deploy@9a86d4c]: (no justification provided) (duration: 01m 06s) | [production] | 
            
  | 21:29 | <mobrovac@tin> | Started deploy [restbase/deploy@9a86d4c]: (no justification provided) | [production] | 
            
  | 21:13 | <hashar> | Gracefully restarting Zuul | [production] | 
            
  | 21:11 | <ppchelko@tin> | Finished deploy [changeprop/deploy@4ba3c59]: Rate-limiter enhancements (duration: 01m 08s) | [production] | 
            
  | 21:10 | <ppchelko@tin> | Started deploy [changeprop/deploy@4ba3c59]: Rate-limiter enhancements | [production] | 
            
  | 21:02 | <demon@tin> | Synchronized php-1.30.0-wmf.5/extensions/CentralAuth/includes/CentralAuthHooks.php: Fix bad method name (duration: 00m 44s) | [production] | 
            
  | 20:37 | <hashar> | Restarting Nodepool. apparently confused in pool tracking and spawning to many Trusty nodes (7 instead of 4) | [production] | 
            
  | 20:02 | <demon@tin> | Synchronized php-1.30.0-wmf.5/includes/api/ApiParse.php: T167826 (duration: 00m 44s) | [production] | 
            
  | 20:00 | <mobrovac@tin> | Finished deploy [restbase/deploy@4c1cdd0]: (no justification provided) (duration: 04m 29s) | [production] | 
            
  | 19:56 | <mobrovac@tin> | Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) | [production] | 
            
  | 19:37 | <Amir1> | restarting ores-related services in scb1001 (T167819) | [production] | 
            
  | 19:24 | <mutante> | scb1001 - killed process 10971 (pdfrendering/electron) | [production] | 
            
  | 19:24 | <demon@tin> | Synchronized php-1.30.0-wmf.5/extensions/CategoryTree/CategoryPageSubclass.php: Fix up variable visibility (duration: 00m 44s) | [production] | 
            
  | 19:12 | <demon@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group0 to wmf.5 | [production] | 
            
  | 19:09 | <mobrovac@tin> | Finished deploy [restbase/deploy@9a86d4c]: (no justification provided) (duration: 07m 33s) | [production] | 
            
  | 19:08 | <mutante> | netmon1002 - reinstallled with stretch, revoked puppet cert, salt key, signing new cert, accepting new key, initial puppet run (T159756) | [production] | 
            
  | 19:01 | <mobrovac@tin> | Started deploy [restbase/deploy@9a86d4c]: (no justification provided) | [production] | 
            
  | 18:56 | <mutante> | reinstalling netmon1002 with stretch - scheduled icinga downtime | [production] | 
            
  | 18:54 | <legoktm> | starting to delete all rows from linter tables on large wikis - T167758 | [production] | 
            
  | 18:48 | <mobrovac@tin> | Finished deploy [restbase/deploy@4c1cdd0]: (no justification provided) (duration: 04m 36s) | [production] | 
            
  | 18:43 | <mobrovac@tin> | Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) | [production] | 
            
  | 18:39 | <mobrovac@tin> | Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) | [production] | 
            
  | 18:37 | <mobrovac@tin> | Finished deploy [restbase/deploy@4c1cdd0]: (no justification provided) (duration: 04m 19s) | [production] | 
            
  | 18:33 | <mobrovac@tin> | Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) | [production] | 
            
  | 18:27 | <demon@tin> | Finished scap: testwiki to wmf.5 + l10n bootstrap (duration: 42m 16s) | [production] | 
            
  | 17:52 | <bblack> | cp4021 reboot for bnx2x modparam change | [production] | 
            
  | 17:50 | <ottomata> | merged removal of x_forwarded_for from all varnishkafka webrequest instances | [production] | 
            
  | 17:45 | <ladsgroup@tin> | Finished deploy [ores/deploy@862aea9]: ORES deploy early June: T167223 (duration: 33m 52s) | [production] | 
            
  | 17:45 | <demon@tin> | Started scap: testwiki to wmf.5 + l10n bootstrap | [production] | 
            
  | 17:42 | <demon@tin> | Pruned MediaWiki: 1.30.0-wmf.2 [keeping static files] (duration: 01m 13s) | [production] | 
            
  | 17:40 | <demon@tin> | Pruned MediaWiki: 1.30.0-wmf.1 [keeping static files] (duration: 05m 10s) | [production] | 
            
  | 17:39 | <bblack> | restart varnish-be on cp2002 (mailbox lag, likely induced by swift traffic testing in codfw) | [production] | 
            
  | 17:11 | <ladsgroup@tin> | Started deploy [ores/deploy@862aea9]: ORES deploy early June: T167223 | [production] | 
            
  | 17:06 | <akosiaris> | rebooting sca2003 for tests | [production] | 
            
  | 16:35 | <moritzm> | upgrading osmium to HHVM 3.18 | [production] | 
            
  | 16:08 | <moritzm> | installing libnl security updates on trusty | [production] | 
            
  | 15:41 | <akosiaris> | upload apertium-spa_1.0.0~r78827-1+wmf to apt.wikimedia.org/jessie-wikimedia/main | [production] | 
            
  | 15:41 | <akosiaris> | upload apertium-ita_0.9.0~r78828-1+wmf to apt.wikimedia.org/jessie-wikimedia/main | [production] | 
            
  | 15:41 | <akosiaris> | upload apertium-fra_1.1.0~r78695-1+wmf to apt.wikimedia.org/jessie-wikimedia/main | [production] | 
            
  | 15:41 | <akosiaris> | upload apertium-cat_2.1.0~r78615-1+wmf to apt.wikimedia.org/jessie-wikimedia/main | [production] | 
            
  | 15:41 | <gehel> | restart of relforge1001 to test https://gerrit.wikimedia.org/r/#/c/358353/ | [production] | 
            
  | 15:09 | <gehel> | applying new GC configuration on elastic1018 - T167636 | [production] | 
            
  | 14:53 | <godog> | update inter-routing for upload to point esams to codfw | [production] | 
            
  | 14:22 | <gehel> | restarting elasticsearch on relforge to validate GC configuration - T167636 | [production] | 
            
  | 14:17 | <ottomata> | stopping puppet on cp1045, testing removal of xff from varnishkafka webrequest data | [production] | 
            
  | 14:14 | <godog> | point upload varnish to swift in codfw - T162609 | [production] | 
            
  | 14:11 | <moritzm> | upgrading mw1299-mw1306 to HHVM 3.18 | [production] | 
            
  | 14:10 | <urandom> | T164865: Restart RESTBase dev; apply range delete probability of 1.0 | [production] |