| 2018-02-23
      
      § | 
    
  | 11:38 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Slowly repool db1076 - T186321 (duration: 01m 13s) | [production] | 
            
  | 11:29 | <marostegui> | Restart mariadb on db1076 for binlog format change - T186321 | [production] | 
            
  | 11:28 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1076 for binlog format change - T186321 (duration: 01m 08s) | [production] | 
            
  | 11:03 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1090 after alter table (duration: 01m 12s) | [production] | 
            
  | 11:02 | <moritzm> | installing kernel updates on mw* in codfw | [production] | 
            
  | 10:30 | <hashar> | releases1001: sudo -u jenkins rm -fR /var/lib/jenkins/jobs/mediawiki-private-nightlies/workspace/BRANCH/REL1_??/mediawiki-snapshot-REL1_??-2018????    # T188080 | [production] | 
            
  | 10:19 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Pool db2090 for the first time (duration: 01m 12s) | [production] | 
            
  | 10:08 | <jynus@tin> | Synchronized wmf-config/db-codfw.php: Pool db2090 for the first time (duration: 01m 12s) | [production] | 
            
  | 10:01 | <elukey> | restart hhvm on mw1230 | [production] | 
            
  | 09:54 | <elukey> | restart hhvm on mw1286 | [production] | 
            
  | 09:50 | <elukey> | restart hhvm on mw1227 | [production] | 
            
  | 08:05 | <marostegui> | MariaDB and kernel upgrade on db1083 | [production] | 
            
  | 07:58 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1083, fully repool db1089 - T162807 (duration: 01m 12s) | [production] | 
            
  | 06:55 | <marostegui> | Reboot db2093 to test /srv auto-mounting | [production] | 
            
  | 06:40 | <marostegui> | Deploy schema change on db1090 - T187089 T185128 T153182 | [production] | 
            
  | 06:38 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1090 for alter table (duration: 01m 13s) | [production] | 
            
  | 05:58 | <mutante> | puppetmaster1001 - signing puppet certs for kafkamon1001/kafkamon2001 - initial puppet runs, adding as role spare (T187901) | [production] | 
            
  | 05:40 | <mutante> | ganeti1004 - initial startup of kafkamon1001 - booting to PXE, installing stretch (T187901) | [production] | 
            
  | 04:56 | <mutante> | ganeti: ganeti2004 - creating new VM kafkamon2001 - vcpus=2,memory=8g,disk=60G, row_A codfw (T187901) | [production] | 
            
  | 04:53 | <mutante> | ganeti: creating new VM kafkamon1001 - vcpus=2,memory=8g,disk=60G, row_A eqiad (T187901) | [production] | 
            
  | 02:46 | <demon@tin> | Finished deploy [gerrit/gerrit@23ebf75]: deploying webhooks plugin (duration: 00m 10s) | [production] | 
            
  | 02:46 | <demon@tin> | Started deploy [gerrit/gerrit@23ebf75]: deploying webhooks plugin | [production] | 
            
  | 02:10 | <demon@tin> | Synchronized docroot/: mw.org docroot moving (duration: 01m 13s) | [production] | 
            
  | 01:45 | <eileen> | update process control process-control config revision is 1605238b2e | [production] | 
            
  | 01:20 | <eileen> | update  civicrm revision changed from aa251f1a93 to a47eafcbad, config revision is c1787646bc | [production] | 
            
  | 01:19 | <demon@tin> | Synchronized static/favicon/: smaller favicons (duration: 01m 12s) | [production] | 
            
  | 01:13 | <demon@tin> | Synchronized wmf-config/InitialiseSettings.php: point mkwikt favicon to en version, dupe (duration: 01m 15s) | [production] | 
            
  | 01:08 | <demon@tin> | Synchronized wmf-config/InitialiseSettings.php: rtl wikibooks logo (duration: 01m 13s) | [production] | 
            
  | 01:06 | <demon@tin> | Synchronized static/favicon/wikibooks-rtl.ico: rtl wikibooks logo (duration: 01m 12s) | [production] | 
            
  | 00:52 | <demon@tin> | Synchronized static/images/project-logos/: new project logos for urdu wikt (duration: 01m 13s) | [production] | 
            
  | 00:37 | <krinkle@tin> | Synchronized docroot/m.wikipedia.org/w/mobilelanding.php: Ia54cd736f30808 - rm use of MW_LANG (duration: 01m 13s) | [production] | 
            
  
    | 2018-02-22
      
      § | 
    
  | 22:33 | <demon@tin> | Synchronized php-1.31.0-wmf.22/includes/filerepo/file/LocalFile.php: Id5cdd8ec (duration: 01m 12s) | [production] | 
            
  | 22:32 | <demon@tin> | Synchronized php-1.31.0-wmf.22/includes/externalstore/: Id5cdd8ec (duration: 01m 12s) | [production] | 
            
  | 22:30 | <demon@tin> | Synchronized php-1.31.0-wmf.22/includes/Storage/: Id5cdd8ec (duration: 01m 13s) | [production] | 
            
  | 22:16 | <maxsem@tin> | Synchronized php-1.31.0-wmf.21/extensions/SyntaxHighlight_GeSHi/: T188019 (duration: 01m 12s) | [production] | 
            
  | 22:14 | <maxsem@tin> | Synchronized php-1.31.0-wmf.22/extensions/SyntaxHighlight_GeSHi/: T188019 (duration: 01m 14s) | [production] | 
            
  | 21:51 | <demon@tin> | Synchronized php-1.31.0-wmf.22/includes/externalstore/: I9334d36e (duration: 01m 15s) | [production] | 
            
  | 21:37 | <dzahn@puppetmaster1001> | conftool action : set/pooled=no; selector: name=wdqs1004.eqiad.wmnet | [production] | 
            
  | 21:11 | <gehel> | powercycling wdqs1004 (complete loss of network) | [production] | 
            
  | 20:39 | <demon@tin> | Synchronized php-1.31.0-wmf.22/includes/libs/objectcache/WANObjectCache.php: betterer logging for cache ttl reduction, Iea029e78 (duration: 01m 13s) | [production] | 
            
  | 19:33 | <XioNoX> | redirecting Facebook bots large source of traffic to codfw ( https://gerrit.wikimedia.org/r/#/c/413446/ ) | [production] | 
            
  | 19:14 | <akosiaris> | rolling restart of eqiad appservers. sudo cumin -b3 -s 30 'A:mw-eqiad' 'restart-hhvm' T188019 | [production] | 
            
  | 19:12 | <twentyafterfour@tin> | Synchronized php-1.31.0-wmf.22/extensions/SyntaxHighlight_GeSHi: deploy https://gerrit.wikimedia.org/r/#/c/413437/ (duration: 01m 13s) | [production] | 
            
  | 19:09 | <twentyafterfour@tin> | Synchronized php-1.31.0-wmf.21/extensions/SyntaxHighlight_GeSHi: deploy https://gerrit.wikimedia.org/r/#/c/413437/ (duration: 01m 13s) | [production] | 
            
  | 19:09 | <twentyafterfour> | syncing https://gerrit.wikimedia.org/r/#/c/413437/ | [production] | 
            
  | 19:03 | <chasemp> | baham:~# authdns-update | [production] | 
            
  | 19:00 | <jynus@tin> | Synchronized wmf-config/db-codfw.php: Repool db2073 (duration: 01m 12s) | [production] | 
            
  | 17:23 | <elukey> | installed linux-perf-4.9 on phab1001 to experiment with perf tracing | [production] | 
            
  | 17:20 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Fully repool db1076 (duration: 01m 12s) | [production] | 
            
  | 17:05 | <XioNoX> | rolling back "redirecting ns2 traffic to radon" | [production] |