| 2016-12-14
      
      § | 
    
  | 11:09 | <mobrovac@tin> | Finished deploy [trending-edits/deploy@c3bc30a]: (no message) (duration: 00m 17s) | [production] | 
            
  | 11:08 | <mobrovac@tin> | Starting deploy [trending-edits/deploy@c3bc30a]: (no message) | [production] | 
            
  | 11:07 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1092 (duration: 00m 48s) | [production] | 
            
  | 10:58 | <jynus> | alter table on db1089 - enwiki T69223 | [production] | 
            
  | 10:54 | <mobrovac@tin> | Finished deploy [trending-edits/deploy@c3bc30a]: (no message) (duration: 00m 21s) | [production] | 
            
  | 10:54 | <mobrovac@tin> | Starting deploy [trending-edits/deploy@c3bc30a]: (no message) | [production] | 
            
  | 10:27 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1089 (duration: 00m 47s) | [production] | 
            
  | 10:26 | <gehel@puppetmaster1001> | conftool action : set/pooled=yes; selector: name=elastic2020.codfw.wmnet | [production] | 
            
  | 10:24 | <mobrovac> | restbase deploying 0c06fb7 on restbase1016 | [production] | 
            
  | 10:01 | <oblivian@puppetmaster1001> | conftool action : set/pooled=yes; selector: name=scb1001.eqiad.wmnet | [production] | 
            
  | 09:59 | <gehel@tin> | Finished deploy [kartotherian/deploy@abc731d]: (no message) (duration: 00m 47s) | [production] | 
            
  | 09:58 | <gehel@tin> | Starting deploy [kartotherian/deploy@abc731d]: (no message) | [production] | 
            
  | 09:53 | <oblivian@puppetmaster1001> | conftool action : set/pooled=no; selector: name=scb1001.eqiad.wmnet | [production] | 
            
  | 09:00 | <volans> | restarted ntp on elastic2020, was with unknown offset on Icinga since ~24h | [production] | 
            
  | 08:48 | <gehel> | kill stuck notification script on maps-test2001 - T145534 | [production] | 
            
  | 08:37 | <gehel> | restarting ferm on elastic2020 | [production] | 
            
  | 08:36 | <gehel@tin> | Finished deploy [kartotherian/deploy@3bd1692]: (no message) (duration: 02m 37s) | [production] | 
            
  | 08:33 | <gehel@tin> | Starting deploy [kartotherian/deploy@3bd1692]: (no message) | [production] | 
            
  | 08:17 | <_joe_> | restbase not starting on restbase1016 due to a failed deploy; masked the systemd unit and depooled from conftool | [production] | 
            
  | 08:08 | <_joe_> | removing parsoids main.log from ruthenium, 39 GB occupied | [production] | 
            
  | 06:44 | <marostegui> | Stop replication db2068 for maintenance - T151552 | [production] | 
            
  | 06:43 | <eileen1> | from 76bc6ba8f005c15410b8785c1d5543154187a17f to 454679d201ed59c76a3905cf3ad5ee2d14fef93f | [production] | 
            
  | 06:43 | <marostegui@tin> | Synchronized wmf-config/db-codfw.php: Depool db2068 - T151552 (duration: 01m 48s) | [production] | 
            
  | 06:35 | <marostegui> | Stop MySQL db2048 for maintenance - T149553 | [production] | 
            
  | 00:21 | <mutante> | gerrit restarting to apply fix for diffusion links (T153130) | [production] | 
            
  | 00:18 | <twentyafterfour@tin> | Synchronized php-1.29.0-wmf.6/extensions/GlobalBlocking/includes/GlobalBlocking.class.php: fix T153153 (duration: 01m 00s) | [production] | 
            
  | 00:12 | <mutante> | cobalt (gerrit) re-enabled puppet | [production] | 
            
  
    | 2016-12-13
      
      § | 
    
  | 23:57 | <twentyafterfour@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group0 wikis to 1.29.0-wmf.6 | [production] | 
            
  | 23:03 | <eileen1> | CiviCRM updated from 4bf38dae0ed99e8584190aafbc224b44fcce8b47 to 76bc6ba8f005c15410b8785c1d5543154187a17f | [production] | 
            
  | 22:56 | <robh> | prometheus1003 was failing to pxe boot over and over (no dhcp entry).  powered off. | [production] | 
            
  | 22:48 | <twentyafterfour@tin> | Synchronized php-1.29.0-wmf.6: (no message) (duration: 07m 23s) | [production] | 
            
  | 22:24 | <papaul> | OS install on ms-fe200[5-8] | [production] | 
            
  | 22:11 | <otto@tin> | Finished deploy [eventstreams/deploy@db5d61f]: (no message) (duration: 00m 49s) | [production] | 
            
  | 22:10 | <otto@tin> | Starting deploy [eventstreams/deploy@db5d61f]: (no message) | [production] | 
            
  | 21:59 | <otto@tin> | Finished deploy [eventstreams/deploy@e1cc638]: (no message) (duration: 00m 21s) | [production] | 
            
  | 21:58 | <otto@tin> | Starting deploy [eventstreams/deploy@e1cc638]: (no message) | [production] | 
            
  | 21:20 | <twentyafterfour@tin> | Finished scap: mediawiki 1.29.0-wmf.6: build l10n and sync the branch to testwikis. refs T152563 (duration: 51m 44s) | [production] | 
            
  | 20:44 | <eileen1> | update CiviCRM from 7c8536c799c6eee774027a949178d44400cff3bc to 4bf38dae0ed99e8584190aafbc224b44fcce8b47 | [production] | 
            
  | 20:29 | <twentyafterfour@tin> | Started scap: mediawiki 1.29.0-wmf.6: build l10n and sync the branch to testwikis. refs T152563 | [production] | 
            
  | 20:00 | <elukey> | uploaded prometheus-apache-exporter 0.3-1 to jessie-wikimedia main | [production] | 
            
  | 19:47 | <godog> | bootstrap cassandra-a on restbase1016 T150964 | [production] | 
            
  | 19:41 | <mutante> | ms-fe200[5-8] mgmt and servers added to DNS | [production] | 
            
  | 19:40 | <andrewbogott> | on labservices1001 and 1002, fixing ID overflow with alter table records modify id BIGINT AUTO_INCREMENT NOT NULL; | [production] | 
            
  | 18:51 | <mobrovac@tin> | Finished deploy [trending-edits/deploy@84db7b8]: (no message) (duration: 00m 22s) | [production] | 
            
  | 18:50 | <mobrovac@tin> | Starting deploy [trending-edits/deploy@84db7b8]: (no message) | [production] | 
            
  | 17:30 | <marostegui@tin> | Synchronized wmf-config/db-codfw.php: Repool db2067 - T151552 (duration: 00m 47s) | [production] | 
            
  | 16:19 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1045 - T148967 (duration: 00m 46s) | [production] | 
            
  | 16:10 | <marostegui> | Shut down db2034 and db2048 for maintenance - T149553 | [production] | 
            
  | 16:03 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1083 with 100% load (duration: 00m 45s) | [production] | 
            
  | 15:48 | <mobrovac> | restbase restarting RB in prod for https://gerrit.wikimedia.org/r/#/c/326529/ | [production] |