| 2018-11-15
      
      ยง | 
    
  | 21:06 | <hashar> | Deleting Nodepool instances on contintcloud T209361 | [production] | 
            
  | 21:05 | <hashar> | Stopped nodepool on labnodepool1001.eqiad.wmnet . Service is no more used. T209361 T209642 | [production] | 
            
  | 20:22 | <mholloway-shell@deploy1001> | Finished deploy [kartotherian/deploy@UNKNOWN]: Fix: Loosen WDQS content-type header check to unbreak maps (T209471) (duration: 03m 55s) | [production] | 
            
  | 20:18 | <mholloway-shell@deploy1001> | Started deploy [kartotherian/deploy@UNKNOWN]: Fix: Loosen WDQS content-type header check to unbreak maps (T209471) | [production] | 
            
  | 20:17 | <mutante> | re-added Chase to pwstore, signed .users file, re-encrypted all pwstore files, git pushed | [production] | 
            
  | 20:15 | <mholloway-shell@deploy1001> | Finished deploy [kartotherian/deploy@48a1e83]: Fix: Loosen WDQS content-type header check to unbreak maps (T209471) (duration: 04m 26s) | [production] | 
            
  | 20:11 | <mholloway-shell@deploy1001> | Started deploy [kartotherian/deploy@48a1e83]: Fix: Loosen WDQS content-type header check to unbreak maps (T209471) | [production] | 
            
  | 20:04 | <urandom> | dropping disused keyspaces -- T208616 | [production] | 
            
  | 20:04 | <catrope@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Enable data collection for UnderstandingFirstDay on cswiki and kowiki (duration: 00m 53s) | [production] | 
            
  | 19:51 | <catrope@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Configure sensitive namespaces for EditorJourney schema (T207307) (duration: 00m 53s) | [production] | 
            
  | 19:49 | <catrope@deploy1001> | Synchronized php-1.33.0-wmf.4/extensions/WikimediaEvents/: cherry-picks for T208773 (duration: 00m 54s) | [production] | 
            
  | 18:55 | <jforrester@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Hot-deploy I26f2dc2e: Don't over-ride default Wikibase string limits (duration: 00m 53s) | [production] | 
            
  | 18:50 | <jforrester@deploy1001> | Synchronized wmf-config/Wikibase.php: Hot-deploy Ib10de2e3: Don't set Wikibase string limits when null (duration: 00m 55s) | [production] | 
            
  | 18:31 | <ladsgroup@deploy1001> | Finished deploy [ores/deploy@dba11e9]: Another small update (duration: 13m 42s) | [production] | 
            
  | 18:18 | <ladsgroup@deploy1001> | Started deploy [ores/deploy@dba11e9]: Another small update | [production] | 
            
  | 18:17 | <ladsgroup@deploy1001> | Finished deploy [ores/deploy@51cdf6b]: T208623 (duration: 14m 41s) | [production] | 
            
  | 18:03 | <ladsgroup@deploy1001> | Started deploy [ores/deploy@51cdf6b]: T208623 | [production] | 
            
  | 17:27 | <anomie@mwmaint1002> | Running refreshExternallinksIndex.php on section 7 wikis in group 2 for T209373. This may cause lag in codfw. | [production] | 
            
  | 17:27 | <anomie@mwmaint1002> | Running refreshExternallinksIndex.php on section 6 wikis in group 2 for T209373. This may cause lag in codfw. | [production] | 
            
  | 17:27 | <anomie@mwmaint1002> | Running refreshExternallinksIndex.php on section 5 wikis in group 2 for T209373. This may cause lag in codfw. | [production] | 
            
  | 17:27 | <anomie@mwmaint1002> | Running refreshExternallinksIndex.php on section 3 wikis in group 2 for T209373. This may cause lag in codfw. | [production] | 
            
  | 17:27 | <anomie@mwmaint1002> | Running refreshExternallinksIndex.php on section 2 wikis in group 2 for T209373. This may cause lag in codfw. | [production] | 
            
  | 17:27 | <anomie@mwmaint1002> | Running refreshExternallinksIndex.php on section 1 wikis in group 2 for T209373. This may cause lag in codfw. | [production] | 
            
  | 17:20 | <gehel> | upgrade prometheus-blazegraph-exporter on all wdqs nodes - T206123 | [production] | 
            
  | 16:41 | <bstorm_> | rebooted labsdb1007 for upgrades | [production] | 
            
  | 16:37 | <andrewbogott> | rebuilding labvirt1015 and cloudvirt1015 | [production] | 
            
  | 14:21 | <anomie@mwmaint1002> | Running refreshExternallinksIndex.php on wikitech for T209373. This may cause lag in codfw. | [production] | 
            
  | 14:21 | <anomie@mwmaint1002> | Running refreshExternallinksIndex.php on section 8 wikis in group 1 for T209373. This may cause lag in codfw. | [production] | 
            
  | 14:21 | <anomie@mwmaint1002> | Running refreshExternallinksIndex.php on section 7 wikis in group 1 for T209373. This may cause lag in codfw. | [production] | 
            
  | 14:21 | <anomie@mwmaint1002> | Running refreshExternallinksIndex.php on section 5 wikis in group 1 for T209373. This may cause lag in codfw. | [production] | 
            
  | 14:21 | <anomie@mwmaint1002> | Running refreshExternallinksIndex.php on section 4 wikis in group 1 for T209373. This may cause lag in codfw. | [production] | 
            
  | 14:21 | <anomie@mwmaint1002> | Running refreshExternallinksIndex.php on section 3 wikis in group 1 for T209373. This may cause lag in codfw. | [production] | 
            
  | 14:20 | <anomie@mwmaint1002> | Running refreshExternallinksIndex.php on section 2 wikis in group 1 for T209373. This may cause lag in codfw. | [production] | 
            
  | 14:07 | <gehel> | plugin and JVM upgrade on elasticsearch / cirrus / eqiad completed - T209293 | [production] | 
            
  | 13:59 | <hashar@deploy1001> | rebuilt and synchronized wikiversions files: all wikis to 1.33.0-wmf.4 | [production] | 
            
  | 13:55 | <XioNoX> | push firewall policies to pfw3-eqiad - T209421 | [production] | 
            
  | 13:50 | <banyek> | Deploy schema change on s6 codfw master, this will generate lag on s6 codfw - T85757 | [production] | 
            
  | 13:48 | <moritzm> | installing qemu security updates (which also backport support for SSBD passthrough) on ganeti clusters | [production] | 
            
  | 13:48 | <moritzm> | installing qemu security updates (which also backport support for SSBD passthrough) | [production] | 
            
  | 13:12 | <mobrovac@deploy1001> | Finished deploy [restbase/deploy@22cb0ec]: Add new wikis to RESTBase - T206777 T205710 T205546 T204477 (duration: 19m 56s) | [production] | 
            
  | 13:00 | <Lucas_WMDE> | EU SWAT finished | [production] | 
            
  | 12:59 | <tarrow@deploy1001> | Synchronized php-1.33.0-wmf.4/extensions/RevisionSlider/modules/ext.RevisionSlider.SliderView.js: [[gerrit:473710]] Fix (accidentally?) reversed blue and yellow lines SWAT T208238 T162119 again (duration: 00m 55s) | [production] | 
            
  | 12:57 | <_joe_> | upping pm.maxworkers to 40 on mw1261 on php7.2-fpm, benchmarking T206341 | [production] | 
            
  | 12:56 | <tarrow@deploy1001> | scap failed: average error rate on 3/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/db09a36be5ed3e81155041f7d46ad040 for details) | [production] | 
            
  | 12:53 | <moritzm> | installing nginx security updates | [production] | 
            
  | 12:52 | <mobrovac@deploy1001> | Started deploy [restbase/deploy@22cb0ec]: Add new wikis to RESTBase - T206777 T205710 T205546 T204477 | [production] | 
            
  | 12:47 | <lucaswerkmeister-wmde@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: [[gerrit:471738|Remove wgWBQualityConstraintsCacheCheckConstraintsResults (T207854)]] (duration: 00m 54s) | [production] | 
            
  | 12:42 | <addshore@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: [[gerrit:473224|Prod: increase Schema.org page split test to 25% sampling]] T208755 (duration: 00m 53s) | [production] | 
            
  | 12:40 | <arturo> | T207377 downtime and reboot labmon1001 | [production] | 
            
  | 12:38 | <lucaswerkmeister-wmde@deploy1001> | Synchronized php-1.33.0-wmf.3/extensions/RevisionSlider/modules/ext.RevisionSlider.SliderView.js: [[gerrit:473710|Fix (accidentally?) reversed blue and yellow lines (T162119, T208238)]] (duration: 00m 54s) | [production] |