| 2017-05-04
      
      ยง | 
    
  | 16:09 | <filippo@tin> | scap aborted: README (duration: 00m 28s) | [production] | 
            
  | 16:09 | <filippo@tin> | Started scap: README | [production] | 
            
  | 16:03 | <urandom> | T160759: restoring default Cassandra tombstone_threshold in eqiad | [production] | 
            
  | 16:00 | <godog> | switch deployment server back to tin.eqiad.wmnet | [production] | 
            
  | 15:57 | <jynus@naos> | Synchronized wmf-config/db-eqiad.php: Remove all read traffic from x1, es2 & es3-master-eqiad (duration: 01m 08s) | [production] | 
            
  | 15:45 | <oblivian@puppetmaster1001> | conftool action : set/pooled=false; selector: dnsdisc=swift-rw,name=codfw | [production] | 
            
  | 15:45 | <bblack> | nginx upgraded to 1.11.10-1+wmf1 on cp1051 (cache_misc) | [production] | 
            
  | 15:42 | <bblack> | nginx upgraded to 1.11.10-1+wmf1 on cp1045 (cache_misc) | [production] | 
            
  | 15:36 | <godog> | run-puppet-agent on cache_upload in codfw/swift for swift a/p in codfw | [production] | 
            
  | 15:34 | <chasemp> | add cwd to acl*procurement-review for phab S4 | [production] | 
            
  | 15:32 | <godog> | run-puppet-agent on cache_upload in codfw/swift for swift a/a | [production] | 
            
  | 15:31 | <oblivian:> | Setting swift-rw in eqiad UP | [production] | 
            
  | 15:31 | <oblivian:> | Setting switft-rw in codfw DOWN | [production] | 
            
  | 15:16 | <marostegui> | Deploy alter table on wikidatawiki.wb_terms - db2059- https://phabricator.wikimedia.org/T162539 https://phabricator.wikimedia.org/T163548 | [production] | 
            
  | 15:15 | <chasemp> | labsdb1003 maintain-views --databases ptwikimedia,pawikisourcewbwikimedia,dtywiki --replace-all --debug T164103 | [production] | 
            
  | 15:14 | <marostegui@naos> | Synchronized wmf-config/db-codfw.php: Repool db2066, depool db2059 - T162539 T163548 (duration: 01m 06s) | [production] | 
            
  | 15:03 | <marostegui@naos> | Synchronized wmf-config/db-eqiad.php: Restore db1070 original weight - T160392 (duration: 00m 57s) | [production] | 
            
  | 14:46 | <oblivian:> | Setting wdqs in codfw UP | [production] | 
            
  | 14:44 | <oblivian:> | Setting restbase-async in eqiad DOWN | [production] | 
            
  | 14:43 | <oblivian:> | Setting restbase in codfw DOWN | [production] | 
            
  | 14:43 | <_joe_> | forcing a puppet run on cache (text,maps, misc) in eqiad/codfw to complete the switchback | [production] | 
            
  | 14:40 | <oblivian:> | Setting restbase in eqiad UP | [production] | 
            
  | 14:39 | <oblivian:> | Setting restbase-async in codfw UP | [production] | 
            
  | 14:36 | <moritzm> | installing mysql-connector-java security updates on hadoop cluster | [production] | 
            
  | 14:35 | <_joe_> | running puppet on varnishes in eqiad (text,misc,maps) to pick up the a/a traffic to services | [production] | 
            
  | 14:29 | <jynus> | dropping and recreating user for maintain-views on labsdb1001 T164103 | [production] | 
            
  | 14:24 | <marostegui@naos> | Synchronized wmf-config/db-eqiad.php: Increase db1070 weight - T160392 (duration: 01m 10s) | [production] | 
            
  | 14:23 | <chasemp> | maintain-meta_p --databases dtywiki,pawikisource,ptwikimedia,wbwikimedia --debug labsdb1003 for T164103 | [production] | 
            
  | 14:16 | <chasemp> | maintain-meta_p --all-databases --purge --debug labsdb1001 for T164103 | [production] | 
            
  | 14:09 | <marostegui@naos> | Synchronized wmf-config/db-eqiad.php: Repool db1070 with less weight - T160392 (duration: 01m 16s) | [production] | 
            
  | 14:03 | <chasemp> | maintain-meta_p --all-databases --purge --debug labsdb1009/1010/1011 for T164103 | [production] | 
            
  | 13:31 | <gehel> | restart services on maps eqiad | [production] | 
            
  | 13:21 | <dereckson@naos> | Synchronized wmf-config/throttle.php: Lift Account registration limit for cywiki for an event / T164482 (duration: 01m 08s) | [production] | 
            
  | 13:18 | <gehel> | restart services on maps codfw | [production] | 
            
  | 13:15 | <gehel> | restart services on maps-test | [production] | 
            
  | 12:42 | <marostegui> | Stop MySQL db1070 for maintenance - T160392 | [production] | 
            
  | 12:40 | <marostegui@naos> | Synchronized wmf-config/db-eqiad.php: Depool db1070 for maintenance - T160392 (duration: 01m 35s) | [production] | 
            
  | 12:28 | <marostegui> | Deploy alter table enwiki.revision on dbstore1001 - T132416 | [production] | 
            
  | 11:56 | <moritzm> | installing mysql-connector-java security updates | [production] | 
            
  | 11:45 | <ema> | starting cache_text upgrades to varnish 4.1.6-1wm1 | [production] | 
            
  | 11:38 | <marostegui@naos> | Synchronized wmf-config/db-eqiad.php: Remove db1022 from config files as it will be decommissioned - T163778 (duration: 01m 06s) | [production] | 
            
  | 11:36 | <marostegui@naos> | Synchronized wmf-config/db-codfw.php: Remove db1022 from config files as it will be decommissioned - T163778 (duration: 01m 25s) | [production] | 
            
  | 10:48 | <moritzm> | installing tomcat security updates | [production] | 
            
  | 10:22 | <elukey> | executed DEL ocg_job_status on rdb1007:6379 (new ocg_job_status hash is stored on the ocg* hosts) - T159850 | [production] | 
            
  | 10:11 | <moritzm> | restarting hhvm on mediawiki canaries to pick up freetype security update | [production] | 
            
  | 10:05 | <ema> | restart varnish-be on cp2024 without RT experiment | [production] | 
            
  | 09:40 | <elukey> | stop kafka on kafka1012 and reboot the host for kernel upgrade | [production] | 
            
  | 09:16 | <joal@naos> | Finished deploy [analytics/refinery@9d35029]: (no justification provided) (duration: 02m 58s) | [production] | 
            
  | 09:13 | <joal@naos> | Started deploy [analytics/refinery@9d35029]: (no justification provided) | [production] | 
            
  | 08:50 | <marostegui@naos> | Synchronized wmf-config/db-codfw.php: Remove db1040 from config files as it will be decommissioned - T164057 (duration: 00m 48s) | [production] |