| 
      
        2017-11-27
      
      §
     | 
  
    
  | 21:59 | 
  <awight@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Reenable ORES on frwiki, ruwiki, and wikidata; T181006 (duration: 00m 45s) | 
  [production] | 
            
  | 21:53 | 
  <awight@tin> | 
  Finished deploy [ores/deploy@e58bfbf]: (codfw) Update ORES for impossible threshold handling and new wikidata editquality model, T179711 T180686 T180450 (duration: 05m 06s) | 
  [production] | 
            
  | 21:47 | 
  <awight@tin> | 
  Started deploy [ores/deploy@e58bfbf]: (codfw) Update ORES for impossible threshold handling and new wikidata editquality model, T179711 T180686 T180450 | 
  [production] | 
            
  | 21:46 | 
  <awight@tin> | 
  Finished deploy [ores/deploy@e58bfbf]: Update ORES for impossible threshold handling and new wikidata editquality model, T179711 T180686 T180450 (duration: 12m 55s) | 
  [production] | 
            
  | 21:33 | 
  <awight@tin> | 
  Started deploy [ores/deploy@e58bfbf]: Update ORES for impossible threshold handling and new wikidata editquality model, T179711 T180686 T180450 | 
  [production] | 
            
  | 21:23 | 
  <awight@tin> | 
  Synchronized php-1.31.0-wmf.8/extensions/ORES: ORES error handling for bad thresholds, T181191 (duration: 00m 46s) | 
  [production] | 
            
  | 21:11 | 
  <awight@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Temporarily disable ORES on wikidata (duration: 00m 45s) | 
  [production] | 
            
  | 21:10 | 
  <awight> | 
  Previous “rollback ORES” was from a stale screen session, no actions were taken today. | 
  [production] | 
            
  | 21:08 | 
  <awight@tin> | 
  Finished deploy [ores/deploy@82a13ae]: Rollback ORES (take 3); 181006 (duration: 9942m 42s) | 
  [production] | 
            
  | 20:21 | 
  <demon@tin> | 
  i lied, that was for wgStyleVersion removal | 
  [production] | 
            
  | 20:20 | 
  <demon@tin> | 
  Synchronized wmf-config/CommonSettings.php: cli sapi fixes (duration: 00m 45s) | 
  [production] | 
            
  | 20:17 | 
  <ejegg> | 
  updated payments-wiki from fea6b375affc629f3dbce71aa348704b307e7a94 to f594dfa763292b2f8785b6aa83d6b50ef8cae64e | 
  [production] | 
            
  | 20:03 | 
  <kaldari@tin> | 
  Synchronized wmf-config/InitialiseSettings-labs.php: (no justification provided) (duration: 00m 45s) | 
  [production] | 
            
  | 20:02 | 
  <kaldari@tin> | 
  Synchronized wmf-config/CommonSettings.php: (no justification provided) (duration: 00m 45s) | 
  [production] | 
            
  | 20:01 | 
  <kaldari@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: (no justification provided) (duration: 00m 46s) | 
  [production] | 
            
  | 19:54 | 
  <bblack> | 
  crN-ulsfo: remove lvs400[1-4] from PyBal BGP neighbors list | 
  [production] | 
            
  | 19:42 | 
  <catrope@tin> | 
  Synchronized php-1.31.0-wmf.8/resources/src/mediawiki.rcfilters/mw.rcfilters.UriProcessor.js: T181100 (duration: 00m 45s) | 
  [production] | 
            
  | 19:41 | 
  <demon@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: babel officewiki thing (duration: 00m 45s) | 
  [production] | 
            
  | 19:39 | 
  <catrope@tin> | 
  Synchronized php-1.31.0-wmf.8/includes/specials/SpecialRecentchangeslinked.php: T181100 (duration: 00m 45s) | 
  [production] | 
            
  | 19:34 | 
  <otto@tin> | 
  Finished deploy [eventlogging/eventbus@3df06ab]: Temp deploy to kafka1001 only to catch bug: T180017 (duration: 00m 12s) | 
  [production] | 
            
  | 19:34 | 
  <otto@tin> | 
  Started deploy [eventlogging/eventbus@3df06ab]: Temp deploy to kafka1001 only to catch bug: T180017 | 
  [production] | 
            
  | 19:21 | 
  <catrope@tin> | 
  Synchronized wmf-config/Wikibase.php: T176903 (duration: 00m 45s) | 
  [production] | 
            
  | 19:17 | 
  <AaronSchulz> | 
  Removed more bogus md5 wanobjecache/ metric from graphite[12]001 | 
  [production] | 
            
  | 19:11 | 
  <catrope@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: T179241 (duration: 00m 45s) | 
  [production] | 
            
  | 19:02 | 
  <volans> | 
  restarted ircecho | 
  [production] | 
            
  | 18:51 | 
  <ejegg> | 
  updated payments-wiki from 6b3019b1f18b4d6cd1705a49e95005124435e3d2 to fea6b375affc629f3dbce71aa348704b307e7a94 | 
  [production] | 
            
  | 18:50 | 
  <dcausse> | 
  elastic/cirrus: reindexing english group2 wikis: T179945 | 
  [production] | 
            
  | 18:46 | 
  <Zoranzoki21> | 
  test | 
  [production] | 
            
  | 18:45 | 
  <volans> | 
  stopped ircecho temporary to avoid spam | 
  [production] | 
            
  | 18:14 | 
  <XioNoX> | 
  pushing v6 gre firewall filter | 
  [production] | 
            
  | 18:12 | 
  <gehel@tin> | 
  Finished deploy [wdqs/wdqs@7ad20f3]: (no justification provided) (duration: 02m 10s) | 
  [production] | 
            
  | 18:11 | 
  <gehel> | 
  deploying blazegraph + GUI updates | 
  [production] | 
            
  | 18:10 | 
  <gehel@tin> | 
  Started deploy [wdqs/wdqs@7ad20f3]: (no justification provided) | 
  [production] | 
            
  | 18:08 | 
  <hoo> | 
  Killed truthy nt dumpers stuck at 100% CPU (from last week) on snapshot1007 (T181385) | 
  [production] | 
            
  | 18:06 | 
  <akosiaris> | 
  poweroff ganeti1005, ganeti1006 T181121 | 
  [production] | 
            
  | 18:05 | 
  <bd808> | 
  Sending Toolforge survey 1 week reminder emails from silver for T177126 | 
  [production] | 
            
  | 16:07 | 
  <bblack> | 
  lvs400x - puppet disabled for https://gerrit.wikimedia.org/r/#/c/393610 | 
  [production] | 
            
  | 15:52 | 
  <herron> | 
  beginning canary cutover/deployment of codfw prometheus servers to codfw puppet 4 puppetmasters | 
  [production] | 
            
  | 15:19 | 
  <chasemp> | 
  disable puppet across cloud things for cleanup | 
  [production] | 
            
  | 15:11 | 
  <herron> | 
  beginning canary cutover/deployment of codfw elasticsearch servers to codfw puppet 4 puppetmasters | 
  [production] | 
            
  | 14:22 | 
  <zeljkof> | 
  EU SWAT finished | 
  [production] | 
            
  | 14:19 | 
  <zfilipin@tin> | 
  Synchronized wmf-config/throttle.php: SWAT: [[gerrit:393442|IP cap lift for Semaine contributive 2017-2018 (T181360)]] (duration: 00m 45s) | 
  [production] | 
            
  | 14:10 | 
  <addshore@tin> | 
  Synchronized php-1.31.0-wmf.8/extensions/AdvancedSearch: SWAT AdvancedSearch T181175 T181222, adjust links in beta section & fix usability of mobile search (duration: 00m 46s) | 
  [production] | 
            
  | 13:31 | 
  <moritzm> | 
  installing nspr security updates on trusty | 
  [production] | 
            
  | 13:22 | 
  <elukey> | 
  remove eventlogging replication support (log database) from dbstore1002 - T156844 | 
  [production] | 
            
  | 13:17 | 
  <moritzm> | 
  updating nginx on francium | 
  [production] | 
            
  | 13:01 | 
  <Pchelolo> | 
  stop cpjobqueue in eqiad for backlog accumulation | 
  [production] | 
            
  | 12:44 | 
  <Pchelolo> | 
  started parsoid linter script to generate load on cpjobqueue (python3 parsoid_reparse.py http://parsoid.discovery.wmnet:8000 --sitematrix --linter-only --skip-closed https://commons.wikimedia.org/w/api.php) | 
  [production] | 
            
  | 12:44 | 
  <addshore@tin> | 
  Synchronized wmf-config/InitialiseSettings-labs.php: [[gerrit:393587]] BETA wmgMonologChannels FileImporter => debug (duration: 01m 01s) | 
  [production] | 
            
  | 12:43 | 
  <jmm@puppetmaster1001> | 
  conftool action : set/pooled=inactive; selector: mw1276.eqiad.wmnet | 
  [production] |