| 2018-05-08
      
      ยง | 
    
  | 19:15 | <twentyafterfour@tin> | Started scap: testwikis wikis to 1.32.0-wmf.3 | [production] | 
            
  | 19:14 | <twentyafterfour> | testwikis to 1.32.0-wmf.3 - https://gerrit.wikimedia.org/r/#/c/431821/ refs T191049 | [production] | 
            
  | 19:11 | <twentyafterfour> | updated mediawiki changelog https://www.mediawiki.org/wiki/MediaWiki_1.32/wmf.3/Changelog refs T191049 | [production] | 
            
  | 18:55 | <mutante> | mw2202, mw2203, mw2204 - reinstall with stretch | [production] | 
            
  | 18:52 | <marostegui> | Manually fail disk #7 on db1073 to get it replaced | [production] | 
            
  | 18:22 | <mutante> | mwmaint1001 - rebooting | [production] | 
            
  | 18:18 | <twentyafterfour> | Branching MediaWiki master to wmf/1.32.0-wmf.3 refs T191049 | [production] | 
            
  | 18:08 | <dzahn@neodymium> | conftool action : set/pooled=yes; selector: name=mw2252.codfw.wmnet | [production] | 
            
  | 18:07 | <andrew@tin> | Finished deploy [horizon/deploy@9245ca9]: rolling out member dashboard (duration: 03m 18s) | [production] | 
            
  | 18:06 | <dzahn@neodymium> | conftool action : set/pooled=yes; selector: name=mw2251.codfw.wmnet | [production] | 
            
  | 18:03 | <andrew@tin> | Started deploy [horizon/deploy@9245ca9]: rolling out member dashboard | [production] | 
            
  | 17:00 | <bawolff> | Clear botpassword throttle for [[User:TaxonBot]] (T194160) | [production] | 
            
  | 16:45 | <thcipriani@tin> | Synchronized README: Testing Scap 3.8.1-1 (duration: 01m 02s) | [production] | 
            
  | 16:43 | <godog> | upload scap 3.8.1-1 - T127762 | [production] | 
            
  | 16:40 | <XioNoX> | re-pooling lvs2001 - T193677 | [production] | 
            
  | 16:36 | <mutante> | mwmaint1001 - reinstalling one more time after proxysql issues are resolved, PXE booting (T192092) | [production] | 
            
  | 16:27 | <mutante> | mw2251,mw2252,mw2201 - reinstall with stretch | [production] | 
            
  | 16:22 | <herron> | cleared low count edac counters on hosts mw2205 dbstore1002 db1051 elastic1029 T183177 | [production] | 
            
  | 16:19 | <urandom> | force (split) compaction of wikipedia_T_mobile__ng_lead.data, restbase1016 - T192689 | [production] | 
            
  | 16:15 | <dzahn@neodymium> | conftool action : set/pooled=yes; selector: name=mw2223.codfw.wmnet | [production] | 
            
  | 16:14 | <dzahn@neodymium> | conftool action : set/pooled=yes; selector: name=mw2222.codfw.wmnet | [production] | 
            
  | 16:10 | <dzahn@neodymium> | conftool action : set/pooled=yes; selector: name=mw2215.codfw.wmnet | [production] | 
            
  | 16:09 | <XioNoX> | failing traffic over lvs2004 - T193677 | [production] | 
            
  | 16:04 | <ppchelko@tin> | Finished deploy [changeprop/deploy@e468d8e]: Allow protocol version negotiation. Codfw only. T167039 (duration: 01m 03s) | [production] | 
            
  | 16:03 | <ppchelko@tin> | Started deploy [changeprop/deploy@e468d8e]: Allow protocol version negotiation. Codfw only. T167039 | [production] | 
            
  | 16:01 | <ppchelko@tin> | Finished deploy [cpjobqueue/deploy@58935d5]: Allow protocol version negotiation. Codfw only. T167039 (duration: 00m 42s) | [production] | 
            
  | 16:01 | <ppchelko@tin> | Started deploy [cpjobqueue/deploy@58935d5]: Allow protocol version negotiation. Codfw only. T167039 | [production] | 
            
  | 15:53 | <mutante> | switching performance.wikimedia.org from graphite to webperf backends - running puppet on cache::misc servers (T158837) | [production] | 
            
  | 15:46 | <demon@tin> | Pruned MediaWiki: 1.32.0-wmf.1 [keeping static files] (duration: 01m 47s) | [production] | 
            
  | 15:30 | <XioNoX> | starting pybal on lvs2001 - T193677 | [production] | 
            
  | 15:26 | <godog> | (un)load edac kernel modules on thumbor1004 to test resetting counters - T183177 | [production] | 
            
  | 15:09 | <XioNoX> | stopping pybal on lvs2001 - T193677 | [production] | 
            
  | 15:06 | <ottomata> | beginnng Kafka upgrade of main-codfw: T167039 | [production] | 
            
  | 14:53 | <XioNoX> | re-enable pybal on lvs2004 - T193677 | [production] | 
            
  | 14:48 | <XioNoX> | disabling pybal on lvs2004 - T193677 | [production] | 
            
  | 14:37 | <mutante> | LDAP: added 'sbailey' to group 'wmf' (T194091) | [production] | 
            
  | 14:19 | <ppchelko@tin> | Started restart [changeprop/deploy@7e86531]: Restart changeprop to try forcing it rebalancing topics | [production] | 
            
  | 14:15 | <mutante> | mw2215,mw2222,mw2223 - reinstalling with stretch | [production] | 
            
  | 13:43 | <zeljkof> | EU SWAT finished | [production] | 
            
  | 13:42 | <zfilipin@tin> | Synchronized php-1.32.0-wmf.2/extensions/Translate: SWAT: [[gerrit:431744|Refactor TranslationUpdateJob to use only primitive types for parameters (T192111)]] (duration: 01m 11s) | [production] | 
            
  | 13:25 | <zfilipin@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:431628|Enable maps i18n everywhere (T191655)]] (duration: 01m 00s) | [production] | 
            
  | 13:14 | <zfilipin@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:430388|Enable AdvancedSearch BetaFeature on all wikis (T193182)]] (duration: 01m 00s) | [production] | 
            
  | 13:02 | <marostegui> | Manually fail disk #9 on db1073 to get it replaced | [production] | 
            
  | 12:20 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Remove db1055 (duration: 00m 59s) | [production] | 
            
  | 12:19 | <moritzm> | reimaging mw2159, mw2160, mw2161 (job runners) to stretch | [production] | 
            
  | 12:18 | <jynus@tin> | Synchronized wmf-config/db-codfw.php: Remove db1055 (duration: 00m 59s) | [production] | 
            
  | 12:17 | <moritzm> | upgrading app servers in beta to wikidiff 1.6.0 (T190717) | [production] | 
            
  | 12:16 | <moritzm> | upgrading app servers in beta to | [production] | 
            
  | 12:02 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Pool db1064 with low load (duration: 00m 59s) | [production] | 
            
  | 11:36 | <marostegui> | Deploy schema change on db1103:3314 - T191519 T188299 T190148 | [production] |