2023-02-27
ยง
|
14:33 |
<bking@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
14:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1127 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P44862 and previous config saved to /var/cache/conftool/dbconfig/20230227-143321-root.json |
[production] |
14:33 |
<jbond@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on idm2001.wikimedia.org with reason: host still been configuered - T320797 |
[production] |
14:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P44861 and previous config saved to /var/cache/conftool/dbconfig/20230227-143311-root.json |
[production] |
14:33 |
<jbond@cumin2002> |
START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on idm2001.wikimedia.org with reason: host still been configuered - T320797 |
[production] |
14:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2180 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P44860 and previous config saved to /var/cache/conftool/dbconfig/20230227-143305-root.json |
[production] |
14:33 |
<jbond@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on idm2001.wikimedia.org with reason: host still been configuered - T320797 |
[production] |
14:33 |
<jbond@cumin2002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on idm2001.wikimedia.org with reason: host still been configuered - T320797 |
[production] |
14:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2146 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P44859 and previous config saved to /var/cache/conftool/dbconfig/20230227-143259-root.json |
[production] |
14:32 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2178 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P44858 and previous config saved to /var/cache/conftool/dbconfig/20230227-143254-root.json |
[production] |
14:32 |
<cmooney@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dse-k8s-worker1006.eqiad.wmnet with reason: host reimage |
[production] |
14:29 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2122 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P44857 and previous config saved to /var/cache/conftool/dbconfig/20230227-142919-root.json |
[production] |
14:28 |
<cmooney@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on dse-k8s-worker1006.eqiad.wmnet with reason: host reimage |
[production] |
14:22 |
<bking@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) |
[production] |
14:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2175 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P44855 and previous config saved to /var/cache/conftool/dbconfig/20230227-142020-root.json |
[production] |
14:18 |
<bking@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
14:18 |
<bking@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) |
[production] |
14:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1127 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P44854 and previous config saved to /var/cache/conftool/dbconfig/20230227-141815-root.json |
[production] |
14:18 |
<bking@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
14:18 |
<bking@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) |
[production] |
14:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P44853 and previous config saved to /var/cache/conftool/dbconfig/20230227-141806-root.json |
[production] |
14:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2180 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P44852 and previous config saved to /var/cache/conftool/dbconfig/20230227-141800-root.json |
[production] |
14:17 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2146 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P44851 and previous config saved to /var/cache/conftool/dbconfig/20230227-141754-root.json |
[production] |
14:17 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2178 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P44850 and previous config saved to /var/cache/conftool/dbconfig/20230227-141749-root.json |
[production] |
14:17 |
<bking@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
14:17 |
<bking@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
14:14 |
<cmooney@cumin1001> |
START - Cookbook sre.hosts.reimage for host dse-k8s-worker1006.eqiad.wmnet with OS bullseye |
[production] |
14:14 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2122 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P44849 and previous config saved to /var/cache/conftool/dbconfig/20230227-141415-root.json |
[production] |
14:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es2022 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P44848 and previous config saved to /var/cache/conftool/dbconfig/20230227-141130-root.json |
[production] |
14:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1022 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P44847 and previous config saved to /var/cache/conftool/dbconfig/20230227-141120-root.json |
[production] |
14:08 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1137 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P44846 and previous config saved to /var/cache/conftool/dbconfig/20230227-140811-root.json |
[production] |
14:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1182 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P44845 and previous config saved to /var/cache/conftool/dbconfig/20230227-140707-root.json |
[production] |
14:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1143 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P44844 and previous config saved to /var/cache/conftool/dbconfig/20230227-140527-root.json |
[production] |
14:05 |
<ladsgroup@deploy1002> |
Synchronized php-1.40.0-wmf.24/extensions/MobileFrontend/includes/MobileContext.php: Completely get rid of responsiveimages removal, part III (T326147) (duration: 07m 36s) |
[production] |
14:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1111 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P44843 and previous config saved to /var/cache/conftool/dbconfig/20230227-140523-root.json |
[production] |
14:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2175 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P44842 and previous config saved to /var/cache/conftool/dbconfig/20230227-140515-root.json |
[production] |
14:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1127 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P44841 and previous config saved to /var/cache/conftool/dbconfig/20230227-140310-root.json |
[production] |
14:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P44840 and previous config saved to /var/cache/conftool/dbconfig/20230227-140301-root.json |
[production] |
14:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2180 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P44839 and previous config saved to /var/cache/conftool/dbconfig/20230227-140255-root.json |
[production] |
14:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2146 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P44838 and previous config saved to /var/cache/conftool/dbconfig/20230227-140249-root.json |
[production] |
14:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2178 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P44837 and previous config saved to /var/cache/conftool/dbconfig/20230227-140244-root.json |
[production] |
13:59 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2122 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P44836 and previous config saved to /var/cache/conftool/dbconfig/20230227-135910-root.json |
[production] |
13:59 |
<ladsgroup@deploy1002> |
ladsgroup: Completely get rid of responsiveimages removal, part III (T326147) synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet |
[production] |
13:58 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db2178 db2146 db2180 T330653', diff saved to https://phabricator.wikimedia.org/P44835 and previous config saved to /var/cache/conftool/dbconfig/20230227-135856-root.json |
[production] |
13:58 |
<moritzm> |
restarting apache on mw canaries to pick up apr-util updates |
[production] |
13:56 |
<ladsgroup@deploy1002> |
Synchronized php-1.40.0-wmf.24/extensions/MobileFrontend/includes/MobileFrontendHooks.php: Completely get rid of responsiveimages removal, part II (T326147) (duration: 07m 24s) |
[production] |
13:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es2022 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P44834 and previous config saved to /var/cache/conftool/dbconfig/20230227-135625-root.json |
[production] |
13:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1022 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P44833 and previous config saved to /var/cache/conftool/dbconfig/20230227-135615-root.json |
[production] |
13:53 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1137 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P44832 and previous config saved to /var/cache/conftool/dbconfig/20230227-135306-root.json |
[production] |
13:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1137 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P44831 and previous config saved to /var/cache/conftool/dbconfig/20230227-135225-root.json |
[production] |