| 2016-12-14
      
      § | 
    
  | 08:48 | <gehel> | kill stuck notification script on maps-test2001 - T145534 | [production] | 
            
  | 08:37 | <gehel> | restarting ferm on elastic2020 | [production] | 
            
  | 08:36 | <gehel@tin> | Finished deploy [kartotherian/deploy@3bd1692]: (no message) (duration: 02m 37s) | [production] | 
            
  | 08:33 | <gehel@tin> | Starting deploy [kartotherian/deploy@3bd1692]: (no message) | [production] | 
            
  | 08:17 | <_joe_> | restbase not starting on restbase1016 due to a failed deploy; masked the systemd unit and depooled from conftool | [production] | 
            
  | 08:08 | <_joe_> | removing parsoids main.log from ruthenium, 39 GB occupied | [production] | 
            
  | 06:44 | <marostegui> | Stop replication db2068 for maintenance - T151552 | [production] | 
            
  | 06:43 | <eileen1> | from 76bc6ba8f005c15410b8785c1d5543154187a17f to 454679d201ed59c76a3905cf3ad5ee2d14fef93f | [production] | 
            
  | 06:43 | <marostegui@tin> | Synchronized wmf-config/db-codfw.php: Depool db2068 - T151552 (duration: 01m 48s) | [production] | 
            
  | 06:35 | <marostegui> | Stop MySQL db2048 for maintenance - T149553 | [production] | 
            
  | 00:21 | <mutante> | gerrit restarting to apply fix for diffusion links (T153130) | [production] | 
            
  | 00:18 | <twentyafterfour@tin> | Synchronized php-1.29.0-wmf.6/extensions/GlobalBlocking/includes/GlobalBlocking.class.php: fix T153153 (duration: 01m 00s) | [production] | 
            
  | 00:12 | <mutante> | cobalt (gerrit) re-enabled puppet | [production] | 
            
  
    | 2016-12-13
      
      § | 
    
  | 23:57 | <twentyafterfour@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group0 wikis to 1.29.0-wmf.6 | [production] | 
            
  | 23:03 | <eileen1> | CiviCRM updated from 4bf38dae0ed99e8584190aafbc224b44fcce8b47 to 76bc6ba8f005c15410b8785c1d5543154187a17f | [production] | 
            
  | 22:56 | <robh> | prometheus1003 was failing to pxe boot over and over (no dhcp entry).  powered off. | [production] | 
            
  | 22:48 | <twentyafterfour@tin> | Synchronized php-1.29.0-wmf.6: (no message) (duration: 07m 23s) | [production] | 
            
  | 22:24 | <papaul> | OS install on ms-fe200[5-8] | [production] | 
            
  | 22:11 | <otto@tin> | Finished deploy [eventstreams/deploy@db5d61f]: (no message) (duration: 00m 49s) | [production] | 
            
  | 22:10 | <otto@tin> | Starting deploy [eventstreams/deploy@db5d61f]: (no message) | [production] | 
            
  | 21:59 | <otto@tin> | Finished deploy [eventstreams/deploy@e1cc638]: (no message) (duration: 00m 21s) | [production] | 
            
  | 21:58 | <otto@tin> | Starting deploy [eventstreams/deploy@e1cc638]: (no message) | [production] | 
            
  | 21:20 | <twentyafterfour@tin> | Finished scap: mediawiki 1.29.0-wmf.6: build l10n and sync the branch to testwikis. refs T152563 (duration: 51m 44s) | [production] | 
            
  | 20:44 | <eileen1> | update CiviCRM from 7c8536c799c6eee774027a949178d44400cff3bc to 4bf38dae0ed99e8584190aafbc224b44fcce8b47 | [production] | 
            
  | 20:29 | <twentyafterfour@tin> | Started scap: mediawiki 1.29.0-wmf.6: build l10n and sync the branch to testwikis. refs T152563 | [production] | 
            
  | 20:00 | <elukey> | uploaded prometheus-apache-exporter 0.3-1 to jessie-wikimedia main | [production] | 
            
  | 19:47 | <godog> | bootstrap cassandra-a on restbase1016 T150964 | [production] | 
            
  | 19:41 | <mutante> | ms-fe200[5-8] mgmt and servers added to DNS | [production] | 
            
  | 19:40 | <andrewbogott> | on labservices1001 and 1002, fixing ID overflow with alter table records modify id BIGINT AUTO_INCREMENT NOT NULL; | [production] | 
            
  | 18:51 | <mobrovac@tin> | Finished deploy [trending-edits/deploy@84db7b8]: (no message) (duration: 00m 22s) | [production] | 
            
  | 18:50 | <mobrovac@tin> | Starting deploy [trending-edits/deploy@84db7b8]: (no message) | [production] | 
            
  | 17:30 | <marostegui@tin> | Synchronized wmf-config/db-codfw.php: Repool db2067 - T151552 (duration: 00m 47s) | [production] | 
            
  | 16:19 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1045 - T148967 (duration: 00m 46s) | [production] | 
            
  | 16:10 | <marostegui> | Shut down db2034 and db2048 for maintenance - T149553 | [production] | 
            
  | 16:03 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1083 with 100% load (duration: 00m 45s) | [production] | 
            
  | 15:48 | <mobrovac> | restbase restarting RB in prod for https://gerrit.wikimedia.org/r/#/c/326529/ | [production] | 
            
  | 15:19 | <mobrovac> | restbase deploy end of 0c06fb7 | [production] | 
            
  | 15:13 | <gehel@tin> | Finished deploy [tilerator/deploy@0fe5a1d]: (no message) (duration: 00m 15s) | [production] | 
            
  | 15:13 | <gehel@tin> | Starting deploy [tilerator/deploy@0fe5a1d]: (no message) | [production] | 
            
  | 15:12 | <hashar> | Added dcausse to Gerrit group "wmf-deployment" so he can Code-Review +2 on the wmf/* branches! | [production] | 
            
  | 15:03 | <mobrovac@tin> | Finished deploy [trending-edits/deploy@5d1eb88]: (no message) (duration: 00m 29s) | [production] | 
            
  | 15:03 | <mobrovac@tin> | Starting deploy [trending-edits/deploy@5d1eb88]: (no message) | [production] | 
            
  | 14:55 | <mobrovac> | restbase deploy start of 0c06fb7 | [production] | 
            
  | 14:47 | <elukey> | testing prometheus-apache-exporter on mw2198 | [production] | 
            
  | 14:40 | <gehel@tin> | Finished deploy [tilerator/deploy@2ae591a]: (no message) (duration: 00m 16s) | [production] | 
            
  | 14:40 | <gehel@tin> | Starting deploy [tilerator/deploy@2ae591a]: (no message) | [production] | 
            
  | 14:31 | <akosiaris@puppetmaster1001> | conftool action : set/pooled=yes; selector: scb1004.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=trendingedits']) | [production] | 
            
  | 14:31 | <akosiaris@puppetmaster1001> | conftool action : set/pooled=yes; selector: scb1003.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=trendingedits']) | [production] | 
            
  | 14:31 | <akosiaris@puppetmaster1001> | conftool action : set/pooled=yes; selector: scb1002.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=trendingedits']) | [production] | 
            
  | 14:31 | <akosiaris@puppetmaster1001> | conftool action : set/pooled=yes; selector: scb1001.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=trendingedits']) | [production] |