| 2020-06-04
      
      ยง | 
    
  | 12:02 | <moritzm> | upgrading mw1276 to PHP 7.2.31 | [production] | 
            
  | 11:59 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Slowly repool db1107', diff saved to https://phabricator.wikimedia.org/P11396 and previous config saved to /var/cache/conftool/dbconfig/20200604-115933-marostegui.json | [production] | 
            
  | 11:53 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: ec07467: wgNamespaceRobotPolicies: thwiki: Add 100 NS to noindex (T253574) (duration: 01m 15s) | [production] | 
            
  | 11:49 | <elukey@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 11:48 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: 338cb90: 1ade16f: Change $wgNamespaceRobotPolicies on Thai wikis (T253578; T253577; T253576; T253575; T253574) (duration: 01m 07s) | [production] | 
            
  | 11:46 | <elukey@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 11:41 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Slowly repool db1107', diff saved to https://phabricator.wikimedia.org/P11395 and previous config saved to /var/cache/conftool/dbconfig/20200604-114149-marostegui.json | [production] | 
            
  | 11:29 | <marostegui> | Compress InnoDB on db1091 before pooling it as new slave on s1 - T254462 | [production] | 
            
  | 11:21 | <hashar@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: [metawiki] Add `centralauth-rename` to WMF OIT staff - T254372 (duration: 01m 08s) | [production] | 
            
  | 11:14 | <elukey@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 11:12 | <elukey@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 11:04 | <jayme@deploy1001> | helmfile [CODFW] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . | [production] | 
            
  | 10:59 | <jayme@deploy1001> | helmfile [CODFW] Ran 'sync' command on namespace 'cxserver' for release 'production' . | [production] | 
            
  | 10:53 | <jayme@deploy1001> | helmfile [CODFW] Ran 'sync' command on namespace 'citoid' for release 'production' . | [production] | 
            
  | 10:53 | <elukey@puppetmaster1001> | conftool action : set/pooled=no; selector: name=druid1004.eqiad.wmnet | [production] | 
            
  | 10:46 | <marostegui> | Deploy schema change on s3 (only testwiki) eqiad - T238966 | [production] | 
            
  | 10:42 | <marostegui> | Deploy schema change on s3 (only testwiki) codfw - T238966 | [production] | 
            
  | 10:41 | <jbond42> | deployed new version of puppet-merge revert is https://gerrit.wikimedia.org/r/c/operations/puppet/+/602329 | [production] | 
            
  | 09:57 | <jayme@deploy1001> | helmfile [STAGING] Ran 'sync' command on namespace 'wikifeeds' for release 'staging' . | [production] | 
            
  | 09:56 | <akosiaris@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 09:56 | <akosiaris@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 09:55 | <akosiaris@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 09:55 | <akosiaris@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 09:55 | <akosiaris@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 09:54 | <jayme@deploy1001> | helmfile [STAGING] Ran 'sync' command on namespace 'cxserver' for release 'staging' . | [production] | 
            
  | 09:53 | <akosiaris@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 09:51 | <akosiaris@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 09:51 | <akosiaris@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 09:50 | <akosiaris@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 09:50 | <akosiaris@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 09:50 | <akosiaris@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 09:50 | <akosiaris@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 09:50 | <akosiaris@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 09:50 | <akosiaris@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 09:50 | <akosiaris@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 09:50 | <akosiaris@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 09:50 | <akosiaris@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 09:50 | <akosiaris@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 09:50 | <akosiaris@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 09:50 | <akosiaris@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 09:48 | <akosiaris@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 09:48 | <akosiaris@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 09:48 | <akosiaris@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 09:48 | <akosiaris@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 09:48 | <akosiaris@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 09:48 | <akosiaris@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 09:46 | <jayme@deploy1001> | helmfile [STAGING] Ran 'sync' command on namespace 'citoid' for release 'staging' . | [production] | 
            
  | 09:42 | <jmm@cumin2001> | END (FAIL) - Cookbook sre.cassandra.roll-restart (exit_code=99) | [production] | 
            
  | 09:42 | <jmm@cumin2001> | START - Cookbook sre.cassandra.roll-restart | [production] | 
            
  | 09:41 | <jmm@cumin2001> | END (FAIL) - Cookbook sre.cassandra.roll-restart (exit_code=99) | [production] |