4351-4400 of 10000 results (72ms)
2019-07-25 ยง
15:52 <jeh> rebooting cloudstore1008.wikimedia.org for updates T224228 [production]
15:41 <jeh> rebooting cloudstore1009.wikimedia.org for updates T224228 [production]
15:41 <nuria@deploy1001> Finished deploy [analytics/refinery@f310917]: deploying refinery - migrations to hive2 actions (duration: 13m 40s) [production]
15:35 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
15:35 <robh@cumin1001> START - Cookbook sre.hosts.decommission [production]
15:35 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
15:35 <robh@cumin1001> START - Cookbook sre.hosts.decommission [production]
15:32 <jforrester@deploy1001> Synchronized wmf-config/CommonSettings.php: Remove redundant wgResourceLoaderStorageEnabled override (duration: 00m 50s) [production]
15:27 <nuria@deploy1001> Started deploy [analytics/refinery@f310917]: deploying refinery - migrations to hive2 actions [production]
15:14 <moritzm> installing patch security updates for jessie [production]
15:09 <jeh> rebooting labstore1004.eqiad.wmnet for updates T224228 [production]
14:42 <ebernhardson@deploy1001> Finished deploy [wikimedia/discovery/analytics@87b25f2]: Convert oozie actions from hive to hive2 (duration: 00m 19s) [production]
14:42 <ebernhardson@deploy1001> Started deploy [wikimedia/discovery/analytics@87b25f2]: Convert oozie actions from hive to hive2 [production]
14:22 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
14:22 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'coredns' . [production]
14:22 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . [production]
14:06 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
14:06 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'coredns' . [production]
14:06 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . [production]
14:02 <moritzm> installing Java security updates on Druid servers [production]
13:52 <moritzm> installing Java security updates on AQS, Hadoop and Kafka/Jumbo servers [production]
13:49 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
13:49 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'coredns' . [production]
13:49 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . [production]
13:42 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
13:42 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'coredns' . [production]
13:42 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . [production]
13:39 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
13:39 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'coredns' . [production]
13:39 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . [production]
13:39 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . [production]
13:38 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . [production]
13:35 <robh> cloudvirt1015 offline for ram swap via T220853 [production]
13:20 <root@> helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . [production]
13:19 <fsero> recreating clusterrole deploy from helmfile in staging [production]
13:09 <marostegui> Drop abuse_filter_log.afl_log_id in s5 eqiad - T226851 [production]
13:04 <liw@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.34.0-wmf.15 [production]
12:49 <marostegui> Drop abuse_filter_log.afl_log_id in s4 codfw (lag will appear on codfw) - T226851 [production]
11:53 <marostegui> Compress s3 wikis on labsdb1010 - T222978 [production]
11:03 <arturo> update stretch-wikimedia/thirdparty/kubeadm-k8s on install1002 for T215531 (kubeadm 1.15.1) [production]
10:53 <moritzm> rebooting cloudvirt2003-dev [production]
10:52 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:52 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
10:35 <moritzm> rebooting cloudvirt1024 for kernel update [production]
10:35 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:35 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
09:21 <marostegui> Failover m1 from dbproxy1006 to dbproxy1001 - T227139 [production]
08:54 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
08:54 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
08:54 <moritzm> rebooting cloudvirt2001-dev [production]