4901-4950 of 10000 results (34ms)
2020-07-14 ยง
14:26 <oblivian@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'cxserver' for release 'production' . [production]
14:23 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
14:21 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
14:20 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
14:18 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
14:18 <jmm@cumin2001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) [production]
14:18 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
14:14 <oblivian@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'cxserver' for release 'production' . [production]
14:13 <andrewbogott> upgrading wikitech-static to mw 1.34.2 [production]
14:11 <oblivian@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'cxserver' for release 'staging' . [production]
14:05 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
14:01 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
13:42 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.change-distro (exit_code=0) [production]
13:28 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1112', diff saved to https://phabricator.wikimedia.org/P11900 and previous config saved to /var/cache/conftool/dbconfig/20200714-132823-marostegui.json [production]
13:27 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1130', diff saved to https://phabricator.wikimedia.org/P11899 and previous config saved to /var/cache/conftool/dbconfig/20200714-132742-marostegui.json [production]
13:27 <jbond@cumin1001> conftool action : set/pooled=yes; selector: name=dns1001.wikimedia.org [production]
13:24 <jbond42> reboot dns1001 [production]
13:23 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:23 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:22 <jbond@cumin1001> conftool action : set/pooled=no; selector: name=dns1001.wikimedia.org [production]
13:22 <jbond@cumin1001> conftool action : set/pooled=yes; selector: name=dns1002.wikimedia.org [production]
13:18 <jbond42> reboot dns1002 [production]
13:18 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:18 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:18 <jbond@cumin1001> conftool action : set/pooled=no; selector: name=dns1002.wikimedia.org [production]
13:16 <jbond@cumin1001> conftool action : set/pooled=yes; selector: name=dns2002.wikimedia.org [production]
13:13 <jbond42> reboot dns2002 [production]
13:13 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:13 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:13 <jbond@cumin1001> conftool action : set/pooled=no; selector: name=dns2002.wikimedia.org [production]
13:13 <jbond@cumin1001> conftool action : set/pooled=yes; selector: name=dns2001.wikimedia.org [production]
13:10 <jbond42> reboot dns2001 [production]
13:10 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:10 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:10 <jbond@cumin1001> conftool action : set/pooled=no; selector: name=dns2001.wikimedia.org [production]
13:09 <elukey@cumin1001> START - Cookbook sre.hadoop.change-distro [production]
13:06 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.stop-cluster (exit_code=0) [production]
13:01 <jbond42> rebooting dns3002 [production]
13:01 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:01 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:58 <elukey@cumin1001> START - Cookbook sre.hadoop.stop-cluster [production]
12:57 <oblivian@deploy1001> Synchronized wmf-config/InitialiseSettings.php: revert forcehttps after fixing T257887 (duration: 01m 02s) [production]
12:31 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.change-distro (exit_code=0) [production]
12:24 <jbond42> route ns0.wikimedia.org to codfw for reboot [production]
12:20 <moritzm> installing xen security updates (client-side tools/libs) [production]
12:19 <jbond42> re-enable puppet fleet [production]
12:07 <jbond42> disable puppet fleet wide to reboot puppetdb's [production]
12:07 <jbond42> disable puppet ro reboot puppetdb's [production]
12:01 <jforrester@deploy1001> rebuilt and synchronized wikiversions files: group0 wikis to 1.35.0-wmf.41 [production]
11:36 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1130 for query plan checks T238966 ', diff saved to https://phabricator.wikimedia.org/P11898 and previous config saved to /var/cache/conftool/dbconfig/20200714-113612-marostegui.json [production]