| 2020-10-30
      
      ยง | 
    
  | 23:35 | <foks> | removing two files for legal compliance | [production] | 
            
  | 23:32 | <mutante> | adding query.wikidata.org to TLS cert for webserver-misc-apps.discovery.wmnet T266702 | [production] | 
            
  | 23:05 | <robh@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 23:04 | <robh@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 23:04 | <robh@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 23:03 | <robh@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 23:03 | <robh@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 23:03 | <robh@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 23:03 | <robh@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 23:02 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 23:02 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 23:02 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 23:01 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 23:01 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 23:01 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 23:01 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 21:02 | <jiji@cumin2001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 21:00 | <jiji@cumin2001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 20:59 | <mutante> | mw1267,mw1268 - scap pull and repool - back to prod - T266164 | [production] | 
            
  | 20:57 | <dzahn@cumin1001> | conftool action : set/pooled=yes; selector: name=mw1267.eqiad.wmnet | [production] | 
            
  | 20:57 | <dzahn@cumin1001> | conftool action : set/pooled=yes; selector: name=mw1268.eqiad.wmnet | [production] | 
            
  | 20:56 | <mutante> | mw1267,mw1268 - scap pull | [production] | 
            
  | 20:33 | <robh@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 20:32 | <robh@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 20:32 | <robh@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 20:32 | <robh@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 20:32 | <robh@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 20:32 | <robh@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 20:32 | <robh@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 20:32 | <robh@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 20:32 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 20:31 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 20:31 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 20:31 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 20:31 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 20:31 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 20:31 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 20:31 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 20:06 | <robh@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 20:04 | <robh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 18:48 | <cdanis> | the above scap began (and mostly finished) several minutes ago but is hanging on a couple hosts down for maintenance | [production] | 
            
  | 18:48 | <cdanis@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: lower frwiki featured feeds limit 1a41ef634 T266865 (duration: 05m 14s) | [production] | 
            
  | 18:47 | <cdanis> | โ๏ธ cdanis@deploy1001.eqiad.wmnet /srv/mediawiki-staging ๐โ scap sync-file wmf-config/InitialiseSettings.php 'lower frwiki featured feeds limit 1a41ef634 T266865' | [production] | 
            
  | 18:27 | <hashar@deploy1001> | Finished deploy [integration/docroot@c35e5e9]: Add ECS to doc.wikimedia.org index (duration: 00m 06s) | [production] | 
            
  | 18:27 | <hashar@deploy1001> | Started deploy [integration/docroot@c35e5e9]: Add ECS to doc.wikimedia.org index | [production] | 
            
  | 17:38 | <dzahn@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 17:36 | <dzahn@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 17:22 | <dzahn@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 17:22 | <dzahn@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 17:19 | <effie> | disable puppet on mc1036 and mc2036 - T252391 | [production] |