| 
      
        2016-12-01
      
      ยง
     | 
  
    
  | 20:55 | 
  <otto@tin> | 
  Starting deploy [eventlogging/eventbus@948765d]: accept api_version parameter | 
  [production] | 
            
  | 20:53 | 
  <otto@tin> | 
  Finished deploy [eventlogging/eventbus@948765d]: (no message) (duration: 00m 08s) | 
  [production] | 
            
  | 20:53 | 
  <otto@tin> | 
  Starting deploy [eventlogging/eventbus@948765d]: (no message) | 
  [production] | 
            
  | 20:27 | 
  <ottomata> | 
  bouncing kafka broker on kafka1018 to test config changes to eventlogging analytics kafka clients | 
  [production] | 
            
  | 20:03 | 
  <demon@tin> | 
  rebuilt wikiversions.php and synchronized wikiversions files: group2 to wmf.4 | 
  [production] | 
            
  | 19:51 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool pc1006 (duration: 00m 46s) | 
  [production] | 
            
  | 19:48 | 
  <filippo@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: name=mw2092.codfw.wmnet | 
  [production] | 
            
  | 19:45 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool pc1006 (duration: 00m 44s) | 
  [production] | 
            
  | 19:34 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool pc1005 (duration: 00m 46s) | 
  [production] | 
            
  | 19:26 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool pc1005 (duration: 00m 45s) | 
  [production] | 
            
  | 19:17 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool pc1004 (duration: 00m 44s) | 
  [production] | 
            
  | 19:08 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool pc1004 (duration: 00m 45s) | 
  [production] | 
            
  | 18:49 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool es1019 (duration: 00m 45s) | 
  [production] | 
            
  | 18:20 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool es1015 (duration: 00m 45s) | 
  [production] | 
            
  | 18:15 | 
  <jynus> | 
  mysql restart and general upgrade for pc2006 T152029 | 
  [production] | 
            
  | 18:02 | 
  <jynus> | 
  mysql restart and general upgrade for pc2005 T152029 | 
  [production] | 
            
  | 17:23 | 
  <mobrovac@tin> | 
  Finished deploy [electron-render/deploy@d6f7044]: (no message) (duration: 01m 02s) | 
  [production] | 
            
  | 17:22 | 
  <mobrovac@tin> | 
  Starting deploy [electron-render/deploy@d6f7044]: (no message) | 
  [production] | 
            
  | 17:15 | 
  <jynus> | 
  mysql restart and general upgrade for pc2004 T152029 | 
  [production] | 
            
  | 17:05 | 
  <paravoid> | 
  cr1-eqiad: re-enabling ae4 and its members (links to asw2-d-eqiad) | 
  [production] | 
            
  | 17:01 | 
  <otto@tin> | 
  Finished deploy [eventlogging/analytics@948765d]: (no message) (duration: 00m 03s) | 
  [production] | 
            
  | 17:01 | 
  <otto@tin> | 
  Starting deploy [eventlogging/analytics@948765d]: (no message) | 
  [production] | 
            
  | 16:58 | 
  <jynus> | 
  mysql restart for es1019 T151995 | 
  [production] | 
            
  | 16:54 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool es1019 (duration: 00m 45s) | 
  [production] | 
            
  | 16:29 | 
  <chasemp> | 
  labsdb: maintain-views --databases fiwikivoyage --debug | 
  [production] | 
            
  | 16:19 | 
  <jynus> | 
  mysql restart for es1015 T151995 | 
  [production] | 
            
  | 16:12 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool es1018; depool es1015 (duration: 01m 00s) | 
  [production] | 
            
  | 15:56 | 
  <paravoid> | 
  rebooting asw2-d-eqiad again | 
  [production] | 
            
  | 15:51 | 
  <marostegui> | 
  Deploy alter table wikidatawiki.revision in dbstore2002 -T150644 | 
  [production] | 
            
  | 15:38 | 
  <marostegui> | 
  Stopping mysql and shutting down db2048 for maintenance - T149553 | 
  [production] | 
            
  | 15:08 | 
  <marostegui> | 
  DNS change for es2 and es3 after the master switchovers - T151995 | 
  [production] | 
            
  | 15:06 | 
  <gehel> | 
  upgrading logstash to Java 8, including rolling restart - T151325 | 
  [production] | 
            
  | 14:52 | 
  <hashar> | 
  Nodepool / CI are processing again | 
  [production] | 
            
  | 14:48 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: switchover es3 master (eqiad) es1019 -> es1014 (duration: 00m 44s) | 
  [production] | 
            
  | 14:46 | 
  <elukey> | 
  restarting kafka on kafka100[123] (EventBus) for openjdk upgrades | 
  [production] | 
            
  | 14:39 | 
  <zeljkof> | 
  EU SWAT finished | 
  [production] | 
            
  | 14:38 | 
  <zfilipin@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:324401|Allow contentadmin and sysop to add/remove autopatrolled users on Wikitech]] (duration: 00m 50s) | 
  [production] | 
            
  | 14:35 | 
  <mobrovac> | 
  restbase deployed 91551bf | 
  [production] | 
            
  | 14:19 | 
  <elukey> | 
  restarting kafka also on kafka2003 | 
  [production] | 
            
  | 14:17 | 
  <elukey> | 
  restarting kafka on kafka200[12] for openjdk upgrades | 
  [production] | 
            
  | 14:14 | 
  <paravoid> | 
  rebooting asw2-d-eqiad | 
  [production] | 
            
  | 13:57 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1070 - T148967 (duration: 00m 45s) | 
  [production] | 
            
  | 13:52 | 
  <jynus> | 
  changing es3 eqiad replication topology in preparation for master switchover | 
  [production] | 
            
  | 13:47 | 
  <hashar> | 
  Nodepool is out of instances due to OpenStack API spurting a nova.exception.ImageNotAuthorized  HTTP 500 | 
  [production] | 
            
  | 13:21 | 
  <paravoid> | 
  Upgrading asw2-d-eqiad to JunOS 15.1R5 (T133387) | 
  [production] | 
            
  | 13:19 | 
  <paravoid> | 
  cr1-eqiad: setting ae4 and its members (links to asw2-d-eqiad) to disable | 
  [production] | 
            
  | 12:49 | 
  <reedy@tin> | 
  Synchronized php-1.29.0-wmf.3/api.php: Remove oris bandaid T151702 (duration: 00m 46s) | 
  [production] | 
            
  | 10:39 | 
  <ariel@tin> | 
  Finished deploy [dumps/dumps@a3801fa]: second try on db_user fixup (duration: 00m 01s) | 
  [production] | 
            
  | 10:39 | 
  <ariel@tin> | 
  Starting deploy [dumps/dumps@a3801fa]: second try on db_user fixup | 
  [production] | 
            
  | 10:25 | 
  <elukey> | 
  removed --debug flag to the puppet compiler output | 
  [production] |