| 2018-04-17
      
      § | 
    
  | 14:36 | <ariel@tin> | Started deploy [dumps/dumps@1073d75]: more exception logging from xmlstream | [production] | 
            
  | 14:30 | <mobrovac@tin> | Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 50% of anons for enwiki - T191101 (duration: 00m 58s) | [production] | 
            
  | 14:25 | <mobrovac@tin> | Synchronized php-1.31.0-wmf.29/extensions/EventBus/includes/JobQueueEventBus.php: Support per-event dispatch of events, file 3/3 - T191464 (duration: 03m 07s) | [production] | 
            
  | 14:23 | <jynus> | start es1017 reimage | [production] | 
            
  | 14:22 | <mobrovac@tin> | Synchronized php-1.31.0-wmf.29/extensions/EventBus/includes/EventBus.php: Support per-event dispatch of events, file 2/3 - T191464 (duration: 03m 06s) | [production] | 
            
  | 14:16 | <mobrovac@tin> | Synchronized php-1.31.0-wmf.29/extensions/EventBus/extension.json: Support per-event dispatch of events, file 1/3 - T191464 (duration: 03m 00s) | [production] | 
            
  | 14:08 | <vgutierrez> | Depool and reimage lvs3004 as stretch - T191897 | [production] | 
            
  | 13:42 | <mobrovac@tin> | Synchronized php-1.31.0-wmf.29/extensions/EventBus/extension.json: Support per-event dispatch of events, file 1/3 - T191464 (duration: 03m 07s) | [production] | 
            
  | 13:33 | <moritzm> | removed role::mediawiki::imagescaler from deployment-mediawiki05, per watroles the only  use of that role in WMCS | [production] | 
            
  | 13:32 | <moritzm> | removed role::mediawiki::imagescaler from deployment-prep, per watroles the only  use of that role in WMCS | [production] | 
            
  | 13:30 | <jynus> | starting backup from db1067, may generate some lag | [production] | 
            
  | 13:26 | <volans> | updating puppet compiler facts | [production] | 
            
  | 13:25 | <elukey> | completed migration of zookeeper on conf200[123] | [production] | 
            
  | 13:15 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1067 (duration: 00m 58s) | [production] | 
            
  | 13:06 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1114 to get it ready for a network cable change (duration: 00m 58s) | [production] | 
            
  | 13:00 | <elukey> | upgrade zookeeper on conf200[123] to 3.4.9~jessie - T182924 | [production] | 
            
  | 12:31 | <mobrovac@tin> | Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 25% of annons on enwiki, take #2 - T191101 (duration: 00m 58s) | [production] | 
            
  | 12:04 | <mobrovac@tin> | Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 25% of annons on enwiki - T191101 (duration: 01m 03s) | [production] | 
            
  | 10:52 | <ema> | lvs100[63] restart pybal to apply https://gerrit.wikimedia.org/r/424553 T188062 | [production] | 
            
  | 10:39 | <ema> | lvs200[63]: restart pybal to apply https://gerrit.wikimedia.org/r/424553 T188062 | [production] | 
            
  | 10:03 | <mobrovac@tin> | Finished deploy [restbase/deploy@e463fcf]: Use keep-alive for connections to AQS (duration: 20m 17s) | [production] | 
            
  | 09:43 | <mobrovac@tin> | Started deploy [restbase/deploy@e463fcf]: Use keep-alive for connections to AQS | [production] | 
            
  | 09:37 | <moritzm> | reimaging mw1280, mw1281, mw1282 (API servers) to stretch | [production] | 
            
  | 09:36 | <moritzm> | reimaging mw1266, mw1267, mw1268 (app servers) to stretch | [production] | 
            
  | 09:17 | <godog> | restart xenon-log on mwlog* - T169249 | [production] | 
            
  | 08:46 | <vgutierrez@neodymium> | conftool action : set/pooled=yes; selector: name=chromium.wikimedia.org,service=pdns_recursor | [production] | 
            
  | 08:19 | <elukey> | restart nrpe-server on kafka2001 (kafka check not defined) | [production] | 
            
  | 08:01 | <moritzm> | rolling restart of HHVM on video scalers to pick up ICU security update | [production] | 
            
  | 07:42 | <moritzm> | installing ICU security updates | [production] | 
            
  | 07:27 | <jynus> | restarting dbstore2001 | [production] | 
            
  | 07:14 | <moritzm> | installing perl security updates on trusty | [production] | 
            
  | 06:48 | <vgutierrez@neodymium> | conftool action : set/pooled=no; selector: name=chromium.wikimedia.org,service=pdns_recursor | [production] | 
            
  | 06:47 | <vgutierrez> | Depool and reimage chromium as stretch - T187090 | [production] | 
            
  | 06:03 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Fully repool db1114 in API - T191996 (duration: 00m 58s) | [production] | 
            
  | 05:52 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Give more API traffic to db1114 (duration: 00m 58s) | [production] | 
            
  | 05:43 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Slowly repool db1114 in API (duration: 00m 58s) | [production] | 
            
  | 05:35 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Restore main traffic original weight for db1114 (duration: 00m 58s) | [production] | 
            
  | 05:24 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Slowly repool db1114 (duration: 00m 58s) | [production] | 
            
  | 05:21 | <marostegui> | Deploy schema change on db1092 - T187089 T185128 T153182 | [production] | 
            
  | 05:19 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1092 for alter table (duration: 00m 58s) | [production] | 
            
  | 05:11 | <marostegui> | Stop MySQL and reboot db1114 to boot up with the new kernel | [production] | 
            
  | 05:10 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1114 (duration: 00m 59s) | [production] | 
            
  | 02:32 | <l10nupdate@tin> | scap sync-l10n completed (1.31.0-wmf.29) (duration: 05m 27s) | [production] | 
            
  | 01:09 | <krinkle@tin> | Synchronized wmf-config/CommonSettings.php: no-op Ib39022b2c37b033d (duration: 01m 00s) | [production] |