| 
      
        2017-08-30
      
      ยง
     | 
  
    
  | 20:22 | 
  <ppchelko@tin> | 
  Finished deploy [changeprop/deploy@8d5fa29]: Start rejecting deduplicated messages (duration: 01m 05s) | 
  [production] | 
            
  | 20:21 | 
  <ppchelko@tin> | 
  Started deploy [changeprop/deploy@8d5fa29]: Start rejecting deduplicated messages | 
  [production] | 
            
  | 20:07 | 
  <urandom> | 
  T169939: Decommission Cassandra: restbase1009-c.eqiad.wmnet | 
  [production] | 
            
  | 19:09 | 
  <ppchelko@tin> | 
  Finished deploy [changeprop/deploy@ff93ea8]: Don't deduplicate retry messages (duration: 01m 17s) | 
  [production] | 
            
  | 19:07 | 
  <ppchelko@tin> | 
  Started deploy [changeprop/deploy@ff93ea8]: Don't deduplicate retry messages | 
  [production] | 
            
  | 19:06 | 
  <demon@tin> | 
  rebuilt wikiversions.php and synchronized wikiversions files: group1 to wmf.16 | 
  [production] | 
            
  | 19:04 | 
  <demon@tin> | 
  Synchronized php: Symlink swap for group1/wmf.16 (duration: 00m 46s) | 
  [production] | 
            
  | 18:11 | 
  <maxsem@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/374399/4 (duration: 00m 47s) | 
  [production] | 
            
  | 17:29 | 
  <urandom> | 
  T169939: Decommission Cassandra: restbase1009-b.eqiad.wmnet | 
  [production] | 
            
  | 17:27 | 
  <hoo> | 
  Updated the Wikidata property suggester with data from Monday's JSON dump and applied the T132839 workarounds | 
  [production] | 
            
  | 17:12 | 
  <ejegg> | 
  disabled creating CiviMail records for Thank You emails | 
  [production] | 
            
  | 16:49 | 
  <XioNoX> | 
  rolling back vrrp priority change for the cr2<->asw-d-codfw link - T174366 | 
  [production] | 
            
  | 16:34 | 
  <XioNoX> | 
  start work on the cr2<->asw-d-codfw link - T174366 | 
  [production] | 
            
  | 16:13 | 
  <XioNoX> | 
  lowering vrrp priority for the cr2<->asw-d-codfw link - T174366 | 
  [production] | 
            
  | 15:56 | 
  <elukey> | 
  re-added analytics1055 among the hdfs/yarn worker after maintenance | 
  [production] | 
            
  | 15:49 | 
  <bblack> | 
  cp1049 - restart varnish backend, mailbox lag | 
  [production] | 
            
  | 15:43 | 
  <urandom> | 
  T169939: Decommission Cassandra: restbase1009-a.eqiad.wmnet | 
  [production] | 
            
  | 15:02 | 
  <jmm@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: mw1294.eqiad.wmnet | 
  [production] | 
            
  | 14:31 | 
  <moritzm> | 
  restarting nginx on sodium to pick up libxml security update | 
  [production] | 
            
  | 14:07 | 
  <elukey> | 
  restart java daemons on druid100[456] for jvm security updates | 
  [production] | 
            
  | 14:04 | 
  <moritzm> | 
  restarting squid on install* servers to pick up libxml security update | 
  [production] | 
            
  | 14:04 | 
  <moritzm> | 
  restarting squid on installurl downloaders to pick up libxml security update | 
  [production] | 
            
  | 14:02 | 
  <moritzm> | 
  restarting apache on krypton to pick up libxml security update | 
  [production] | 
            
  | 13:57 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Restore db1053 original weight - T168661 (duration: 00m 46s) | 
  [production] | 
            
  | 13:55 | 
  <hashar@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: pagePreviews: Scale A/B test bucket sizes by 10 - T172291 (duration: 00m 46s) | 
  [production] | 
            
  | 13:50 | 
  <moritzm> | 
  restarting squid on url downloaders to pick up libxml security update | 
  [production] | 
            
  | 13:34 | 
  <moritzm> | 
  installing libxml2 security updates | 
  [production] | 
            
  | 13:23 | 
  <moritzm> | 
  installing ffmpeg security upadtes | 
  [production] | 
            
  | 13:21 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Increase db1053 weight - T168661 (duration: 00m 46s) | 
  [production] | 
            
  | 13:19 | 
  <hashar> | 
  European SWAT completed | 
  [production] | 
            
  | 13:16 | 
  <gehel> | 
  restarting wdqs-blazegraph and wdqs-updater on all wdqs nodes for logback config change - T172710 | 
  [production] | 
            
  | 13:11 | 
  <hashar@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Enable SandboxLink on cywiki - T173054 (duration: 00m 48s) | 
  [production] | 
            
  | 12:55 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Increase db1053 weight - T168661 (duration: 00m 45s) | 
  [production] | 
            
  | 12:51 | 
  <gehel> | 
  restart wdqs-updater on wdqs2001 for config change | 
  [production] | 
            
  | 12:40 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Increase db1053 weight - T168661 (duration: 00m 47s) | 
  [production] | 
            
  | 12:26 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1053 with low weight - T168661 (duration: 00m 46s) | 
  [production] | 
            
  | 12:05 | 
  <marostegui> | 
  Upgrade MariaDB to 10.0.32 on db1053 - T168661 | 
  [production] | 
            
  | 12:05 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1053 - T168661 (duration: 00m 47s) | 
  [production] | 
            
  | 11:06 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Restore db1055 original weight - T174265 (duration: 00m 46s) | 
  [production] | 
            
  | 10:24 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Give db1055 more traffic - T174265 (duration: 00m 47s) | 
  [production] | 
            
  | 09:49 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Give db1055 more traffic - T174265 (duration: 00m 47s) | 
  [production] | 
            
  | 09:17 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1059 - T168661 (duration: 00m 47s) | 
  [production] | 
            
  | 09:07 | 
  <elukey> | 
  restart all jvm daemons on druid100[123] for security updates | 
  [production] | 
            
  | 08:51 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Give db1055 more traffic - T174265 (duration: 00m 48s) | 
  [production] | 
            
  | 08:06 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Give db1055 more traffic - T174265 (duration: 00m 47s) | 
  [production] | 
            
  | 07:47 | 
  <jmm@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: mw1228.eqiad.wmnet | 
  [production] | 
            
  | 07:47 | 
  <marostegui> | 
  Upgrade MariaDB on db1059 to 10.0.32 - T168661 | 
  [production] | 
            
  | 07:44 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1059 for a MariaDB upgrade - T168661 (duration: 00m 53s) | 
  [production] | 
            
  | 07:14 | 
  <moritzm> | 
  powering down mw1294 for hardware diagnostics (T1674069 | 
  [production] | 
            
  | 07:12 | 
  <jmm@puppetmaster1001> | 
  conftool action : set/pooled=inactive; selector: mw1294.eqiad.wmnet | 
  [production] |