| 2019-12-10
      
      ยง | 
    
  | 16:04 | <akosiaris@deploy1001> | helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . | [production] | 
            
  | 16:04 | <akosiaris@cumin1001> | END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) | [production] | 
            
  | 16:01 | <akosiaris@deploy1001> | helmfile [EQIAD] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . | [production] | 
            
  | 16:00 | <akosiaris@deploy1001> | helmfile [STAGING] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . | [production] | 
            
  | 15:56 | <moritzm> | installing icu updates from stretch 9.11 point release | [production] | 
            
  | 15:54 | <akosiaris@cumin1001> | END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) | [production] | 
            
  | 15:54 | <akosiaris@cumin1001> | START - Cookbook sre.ganeti.makevm | [production] | 
            
  | 15:53 | <akosiaris@cumin1001> | END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) | [production] | 
            
  | 15:45 | <akosiaris@cumin1001> | START - Cookbook sre.ganeti.makevm | [production] | 
            
  | 15:44 | <akosiaris@cumin1001> | START - Cookbook sre.ganeti.makevm | [production] | 
            
  | 15:44 | <akosiaris@cumin1001> | END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) | [production] | 
            
  | 15:34 | <akosiaris@cumin1001> | START - Cookbook sre.ganeti.makevm | [production] | 
            
  | 15:34 | <akosiaris@cumin1001> | END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) | [production] | 
            
  | 15:24 | <akosiaris@cumin1001> | START - Cookbook sre.ganeti.makevm | [production] | 
            
  | 15:24 | <akosiaris@cumin1001> | END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) | [production] | 
            
  | 15:14 | <akosiaris@cumin1001> | START - Cookbook sre.ganeti.makevm | [production] | 
            
  | 15:13 | <akosiaris@cumin1001> | END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) | [production] | 
            
  | 15:04 | <akosiaris@cumin1001> | START - Cookbook sre.ganeti.makevm | [production] | 
            
  | 15:03 | <reedy@deploy1001> | Synchronized wmf-config/wikitech.php: Load OSM and LdapAuth via extension.json T140852 (duration: 00m 55s) | [production] | 
            
  | 15:01 | <moritzm> | installing systemd updates from stretch 9.11 point release | [production] | 
            
  | 14:59 | <reedy@deploy1001> | Synchronized wmf-config/extension-list: Load OSM and LdapAuth via extension.json for messages (duration: 00m 55s) | [production] | 
            
  | 14:55 | <reedy@deploy1001> | Synchronized wmf-config/wikitech.php: T161553 Bye OSM config! (duration: 00m 55s) | [production] | 
            
  | 14:52 | <akosiaris@cumin1001> | END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) | [production] | 
            
  | 14:43 | <akosiaris@cumin1001> | START - Cookbook sre.ganeti.makevm | [production] | 
            
  | 14:42 | <akosiaris@cumin1001> | END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) | [production] | 
            
  | 14:32 | <akosiaris@cumin1001> | START - Cookbook sre.ganeti.makevm | [production] | 
            
  | 14:30 | <akosiaris@cumin1001> | END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) | [production] | 
            
  | 14:30 | <akosiaris@cumin1001> | START - Cookbook sre.ganeti.makevm | [production] | 
            
  | 14:08 | <jbond42> | rolling restart of varnishkafaka-webrequest and varnishkafaka-eventloggin | [production] | 
            
  | 13:54 | <godog> | remove stale puppetmaster2001:/var/run/confd-template/.*.err | [production] | 
            
  | 13:21 | <marostegui> | Compress table db2115 wikishared.cx_corpora on db2115 - T240325 | [production] | 
            
  | 12:36 | <Urbanecm> | EU SWAT done | [production] | 
            
  | 12:36 | <urbanecm@deploy1001> | Synchronized wmf-config/abusefilter.php: SWAT: 80fac66: Enable abusefilter blocking cap at testwiki (duration: 00m 55s) | [production] | 
            
  | 12:32 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: 80fac66: Enable abusefilter blocking cap at testwiki (duration: 00m 55s) | [production] | 
            
  | 12:19 | <nikerabbit@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:555921|Add wiki-for-human-rights CX campaign (T239977)]] (duration: 00m 56s) | [production] | 
            
  | 11:46 | <_joe_> | restarting etcd on conf1005, also etcdmirrormaker | [production] | 
            
  | 11:34 | <jbond42> | rolloing restart of ats servers | [production] | 
            
  | 11:28 | <mbsantos@deploy1001> | Finished deploy [kartotherian/deploy@452b144] (stretch): Update kartotherian-package to f9fb029 (T240227) (duration: 00m 20s) | [production] | 
            
  | 11:27 | <mbsantos@deploy1001> | Started deploy [kartotherian/deploy@452b144] (stretch): Update kartotherian-package to f9fb029 (T240227) | [production] | 
            
  | 11:04 | <_joe_> | restarting pybal on lvs1015, then 1013 and 1014 to pick up the etcd restart | [production] | 
            
  | 10:59 | <_joe_> | restarting pybal on lvs1016, the the other eqiad pybals, to catch up on etcd restart | [production] | 
            
  | 10:55 | <_joe_> | restarting etcd on conf1004 T237362 | [production] | 
            
  | 10:48 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1127 after table defragmentation (duration: 00m 55s) | [production] | 
            
  | 10:36 | <marostegui> | Optimize wikishared.cx_corpora on db2115 (non compressed table) - T183485 | [production] | 
            
  | 10:35 | <marostegui> | Upgrade db1127 | [production] | 
            
  | 10:13 | <moritzm> | stopping slapd on dubnium/pollux following application of the spare role T224557 | [production] | 
            
  | 10:06 | <onimisionipe> | add new disk to RAID array on cloudelastic1002 - T239957 | [production] | 
            
  | 09:51 | <marostegui> | Optimize wikishared. cx_corpora on db1127 - T183485 | [production] | 
            
  | 09:48 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1127 for table defragmentation (duration: 00m 59s) | [production] | 
            
  | 09:20 | <marostegui> | Restart mysql on dbstore1003, 1004 and 1005 for upgrade | [production] |