| 
      
        2019-07-25
      
      ยง
     | 
  
    
  | 15:35 | 
  <robh@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) | 
  [production] | 
            
  | 15:35 | 
  <robh@cumin1001> | 
  START - Cookbook sre.hosts.decommission | 
  [production] | 
            
  | 15:35 | 
  <robh@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) | 
  [production] | 
            
  | 15:35 | 
  <robh@cumin1001> | 
  START - Cookbook sre.hosts.decommission | 
  [production] | 
            
  | 15:32 | 
  <jforrester@deploy1001> | 
  Synchronized wmf-config/CommonSettings.php: Remove redundant wgResourceLoaderStorageEnabled override (duration: 00m 50s) | 
  [production] | 
            
  | 15:27 | 
  <nuria@deploy1001> | 
  Started deploy [analytics/refinery@f310917]: deploying refinery - migrations to hive2 actions | 
  [production] | 
            
  | 15:14 | 
  <moritzm> | 
  installing patch security updates for jessie | 
  [production] | 
            
  | 15:09 | 
  <jeh> | 
  rebooting labstore1004.eqiad.wmnet for updates T224228 | 
  [production] | 
            
  | 14:42 | 
  <ebernhardson@deploy1001> | 
  Finished deploy [wikimedia/discovery/analytics@87b25f2]: Convert oozie actions from hive to hive2 (duration: 00m 19s) | 
  [production] | 
            
  | 14:42 | 
  <ebernhardson@deploy1001> | 
  Started deploy [wikimedia/discovery/analytics@87b25f2]: Convert oozie actions from hive to hive2 | 
  [production] | 
            
  | 14:22 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . | 
  [production] | 
            
  | 14:22 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'coredns' . | 
  [production] | 
            
  | 14:22 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . | 
  [production] | 
            
  | 14:06 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . | 
  [production] | 
            
  | 14:06 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'coredns' . | 
  [production] | 
            
  | 14:06 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . | 
  [production] | 
            
  | 14:02 | 
  <moritzm> | 
  installing Java security updates on Druid servers | 
  [production] | 
            
  | 13:52 | 
  <moritzm> | 
  installing Java security updates on AQS, Hadoop and Kafka/Jumbo servers | 
  [production] | 
            
  | 13:49 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . | 
  [production] | 
            
  | 13:49 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'coredns' . | 
  [production] | 
            
  | 13:49 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . | 
  [production] | 
            
  | 13:42 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . | 
  [production] | 
            
  | 13:42 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'coredns' . | 
  [production] | 
            
  | 13:42 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . | 
  [production] | 
            
  | 13:39 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . | 
  [production] | 
            
  | 13:39 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'coredns' . | 
  [production] | 
            
  | 13:39 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . | 
  [production] | 
            
  | 13:39 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . | 
  [production] | 
            
  | 13:38 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . | 
  [production] | 
            
  | 13:35 | 
  <robh> | 
  cloudvirt1015 offline for ram swap via T220853 | 
  [production] | 
            
  | 13:20 | 
  <root@> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . | 
  [production] | 
            
  | 13:19 | 
  <fsero> | 
  recreating clusterrole deploy from helmfile in staging | 
  [production] | 
            
  | 13:09 | 
  <marostegui> | 
  Drop abuse_filter_log.afl_log_id in s5 eqiad - T226851 | 
  [production] | 
            
  | 13:04 | 
  <liw@deploy1001> | 
  rebuilt and synchronized wikiversions files: all wikis to 1.34.0-wmf.15 | 
  [production] | 
            
  | 12:49 | 
  <marostegui> | 
  Drop abuse_filter_log.afl_log_id in s4 codfw (lag will appear on codfw) - T226851 | 
  [production] | 
            
  | 11:53 | 
  <marostegui> | 
  Compress s3 wikis on labsdb1010 - T222978 | 
  [production] | 
            
  | 11:03 | 
  <arturo> | 
  update stretch-wikimedia/thirdparty/kubeadm-k8s on install1002 for T215531 (kubeadm 1.15.1) | 
  [production] | 
            
  | 10:53 | 
  <moritzm> | 
  rebooting cloudvirt2003-dev | 
  [production] | 
            
  | 10:52 | 
  <jmm@cumin2001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 10:52 | 
  <jmm@cumin2001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 10:35 | 
  <moritzm> | 
  rebooting cloudvirt1024 for kernel update | 
  [production] | 
            
  | 10:35 | 
  <jmm@cumin2001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 10:35 | 
  <jmm@cumin2001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 09:21 | 
  <marostegui> | 
  Failover m1 from dbproxy1006 to dbproxy1001 - T227139 | 
  [production] | 
            
  | 08:54 | 
  <jmm@cumin2001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 08:54 | 
  <jmm@cumin2001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 08:54 | 
  <moritzm> | 
  rebooting cloudvirt2001-dev | 
  [production] | 
            
  | 08:32 | 
  <Urbanecm> | 
  Password reset for SUL user Strejc | 
  [production] | 
            
  | 08:04 | 
  <oblivian@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: cluster=api_appserver,dc=eqiad,name=mw128[0-3].* | 
  [production] | 
            
  | 08:01 | 
  <oblivian@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: cluster=appserver,dc=eqiad,name=mw12(6[89]|7[0-5]).* | 
  [production] |