| 2017-11-27
      
      § | 
    
  | 21:46 | <awight@tin> | Finished deploy [ores/deploy@e58bfbf]: Update ORES for impossible threshold handling and new wikidata editquality model, T179711 T180686 T180450 (duration: 12m 55s) | [production] | 
            
  | 21:33 | <awight@tin> | Started deploy [ores/deploy@e58bfbf]: Update ORES for impossible threshold handling and new wikidata editquality model, T179711 T180686 T180450 | [production] | 
            
  | 21:23 | <awight@tin> | Synchronized php-1.31.0-wmf.8/extensions/ORES: ORES error handling for bad thresholds, T181191 (duration: 00m 46s) | [production] | 
            
  | 21:11 | <awight@tin> | Synchronized wmf-config/InitialiseSettings.php: Temporarily disable ORES on wikidata (duration: 00m 45s) | [production] | 
            
  | 21:10 | <awight> | Previous “rollback ORES” was from a stale screen session, no actions were taken today. | [production] | 
            
  | 21:08 | <awight@tin> | Finished deploy [ores/deploy@82a13ae]: Rollback ORES (take 3); 181006 (duration: 9942m 42s) | [production] | 
            
  | 20:21 | <demon@tin> | i lied, that was for wgStyleVersion removal | [production] | 
            
  | 20:20 | <demon@tin> | Synchronized wmf-config/CommonSettings.php: cli sapi fixes (duration: 00m 45s) | [production] | 
            
  | 20:17 | <ejegg> | updated payments-wiki from fea6b375affc629f3dbce71aa348704b307e7a94 to f594dfa763292b2f8785b6aa83d6b50ef8cae64e | [production] | 
            
  | 20:03 | <kaldari@tin> | Synchronized wmf-config/InitialiseSettings-labs.php: (no justification provided) (duration: 00m 45s) | [production] | 
            
  | 20:02 | <kaldari@tin> | Synchronized wmf-config/CommonSettings.php: (no justification provided) (duration: 00m 45s) | [production] | 
            
  | 20:01 | <kaldari@tin> | Synchronized wmf-config/InitialiseSettings.php: (no justification provided) (duration: 00m 46s) | [production] | 
            
  | 19:54 | <bblack> | crN-ulsfo: remove lvs400[1-4] from PyBal BGP neighbors list | [production] | 
            
  | 19:42 | <catrope@tin> | Synchronized php-1.31.0-wmf.8/resources/src/mediawiki.rcfilters/mw.rcfilters.UriProcessor.js: T181100 (duration: 00m 45s) | [production] | 
            
  | 19:41 | <demon@tin> | Synchronized wmf-config/InitialiseSettings.php: babel officewiki thing (duration: 00m 45s) | [production] | 
            
  | 19:39 | <catrope@tin> | Synchronized php-1.31.0-wmf.8/includes/specials/SpecialRecentchangeslinked.php: T181100 (duration: 00m 45s) | [production] | 
            
  | 19:34 | <otto@tin> | Finished deploy [eventlogging/eventbus@3df06ab]: Temp deploy to kafka1001 only to catch bug: T180017 (duration: 00m 12s) | [production] | 
            
  | 19:34 | <otto@tin> | Started deploy [eventlogging/eventbus@3df06ab]: Temp deploy to kafka1001 only to catch bug: T180017 | [production] | 
            
  | 19:21 | <catrope@tin> | Synchronized wmf-config/Wikibase.php: T176903 (duration: 00m 45s) | [production] | 
            
  | 19:17 | <AaronSchulz> | Removed more bogus md5 wanobjecache/ metric from graphite[12]001 | [production] | 
            
  | 19:11 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: T179241 (duration: 00m 45s) | [production] | 
            
  | 19:02 | <volans> | restarted ircecho | [production] | 
            
  | 18:51 | <ejegg> | updated payments-wiki from 6b3019b1f18b4d6cd1705a49e95005124435e3d2 to fea6b375affc629f3dbce71aa348704b307e7a94 | [production] | 
            
  | 18:50 | <dcausse> | elastic/cirrus: reindexing english group2 wikis: T179945 | [production] | 
            
  | 18:46 | <Zoranzoki21> | test | [production] | 
            
  | 18:45 | <volans> | stopped ircecho temporary to avoid spam | [production] | 
            
  | 18:14 | <XioNoX> | pushing v6 gre firewall filter | [production] | 
            
  | 18:12 | <gehel@tin> | Finished deploy [wdqs/wdqs@7ad20f3]: (no justification provided) (duration: 02m 10s) | [production] | 
            
  | 18:11 | <gehel> | deploying blazegraph + GUI updates | [production] | 
            
  | 18:10 | <gehel@tin> | Started deploy [wdqs/wdqs@7ad20f3]: (no justification provided) | [production] | 
            
  | 18:08 | <hoo> | Killed truthy nt dumpers stuck at 100% CPU (from last week) on snapshot1007 (T181385) | [production] | 
            
  | 18:06 | <akosiaris> | poweroff ganeti1005, ganeti1006 T181121 | [production] | 
            
  | 18:05 | <bd808> | Sending Toolforge survey 1 week reminder emails from silver for T177126 | [production] | 
            
  | 16:07 | <bblack> | lvs400x - puppet disabled for https://gerrit.wikimedia.org/r/#/c/393610 | [production] | 
            
  | 15:52 | <herron> | beginning canary cutover/deployment of codfw prometheus servers to codfw puppet 4 puppetmasters | [production] | 
            
  | 15:19 | <chasemp> | disable puppet across cloud things for cleanup | [production] | 
            
  | 15:11 | <herron> | beginning canary cutover/deployment of codfw elasticsearch servers to codfw puppet 4 puppetmasters | [production] | 
            
  | 14:22 | <zeljkof> | EU SWAT finished | [production] | 
            
  | 14:19 | <zfilipin@tin> | Synchronized wmf-config/throttle.php: SWAT: [[gerrit:393442|IP cap lift for Semaine contributive 2017-2018 (T181360)]] (duration: 00m 45s) | [production] | 
            
  | 14:10 | <addshore@tin> | Synchronized php-1.31.0-wmf.8/extensions/AdvancedSearch: SWAT AdvancedSearch T181175 T181222, adjust links in beta section & fix usability of mobile search (duration: 00m 46s) | [production] | 
            
  | 13:31 | <moritzm> | installing nspr security updates on trusty | [production] | 
            
  | 13:22 | <elukey> | remove eventlogging replication support (log database) from dbstore1002 - T156844 | [production] | 
            
  | 13:17 | <moritzm> | updating nginx on francium | [production] | 
            
  | 13:01 | <Pchelolo> | stop cpjobqueue in eqiad for backlog accumulation | [production] | 
            
  | 12:44 | <Pchelolo> | started parsoid linter script to generate load on cpjobqueue (python3 parsoid_reparse.py http://parsoid.discovery.wmnet:8000 --sitematrix --linter-only --skip-closed https://commons.wikimedia.org/w/api.php) | [production] | 
            
  | 12:44 | <addshore@tin> | Synchronized wmf-config/InitialiseSettings-labs.php: [[gerrit:393587]] BETA wmgMonologChannels FileImporter => debug (duration: 01m 01s) | [production] | 
            
  | 12:43 | <jmm@puppetmaster1001> | conftool action : set/pooled=inactive; selector: mw1276.eqiad.wmnet | [production] | 
            
  | 12:35 | <moritzm> | powercycling mw1276 | [production] | 
            
  | 12:21 | <addshore@tin> | Synchronized wmf-config/CommonSettings-labs.php: [[gerrit:393582]] LABS ONLY Actually call wfLoadExtension for FileExporter & Importer on beta BETA (T181383) (duration: 00m 44s) | [production] | 
            
  | 12:14 | <marostegui> | Stop replication on db1109 to test table rename for s5/s8 failover | [production] |