|
2022-06-24
§
|
| 06:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30066 and previous config saved to /var/cache/conftool/dbconfig/20220624-060545-root.json |
[production] |
| 06:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1169 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30065 and previous config saved to /var/cache/conftool/dbconfig/20220624-060539-root.json |
[production] |
| 06:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1175 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30064 and previous config saved to /var/cache/conftool/dbconfig/20220624-060146-root.json |
[production] |
| 06:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1174 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30063 and previous config saved to /var/cache/conftool/dbconfig/20220624-060141-root.json |
[production] |
| 06:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1172 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30062 and previous config saved to /var/cache/conftool/dbconfig/20220624-060136-root.json |
[production] |
| 05:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1024 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30061 and previous config saved to /var/cache/conftool/dbconfig/20220624-055643-root.json |
[production] |
| 05:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30060 and previous config saved to /var/cache/conftool/dbconfig/20220624-055042-root.json |
[production] |
| 05:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1169 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30059 and previous config saved to /var/cache/conftool/dbconfig/20220624-055035-root.json |
[production] |
| 05:46 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1175 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30058 and previous config saved to /var/cache/conftool/dbconfig/20220624-054642-root.json |
[production] |
| 05:46 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1174 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30057 and previous config saved to /var/cache/conftool/dbconfig/20220624-054637-root.json |
[production] |
| 05:46 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1172 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30056 and previous config saved to /var/cache/conftool/dbconfig/20220624-054632-root.json |
[production] |
| 05:43 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1170 after kernel reboots', diff saved to https://phabricator.wikimedia.org/P30055 and previous config saved to /var/cache/conftool/dbconfig/20220624-054259-root.json |
[production] |
| 05:41 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1024 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30054 and previous config saved to /var/cache/conftool/dbconfig/20220624-054139-root.json |
[production] |
| 05:36 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1170 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30053 and previous config saved to /var/cache/conftool/dbconfig/20220624-053652-root.json |
[production] |
| 05:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30052 and previous config saved to /var/cache/conftool/dbconfig/20220624-053538-root.json |
[production] |
| 05:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1169 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30051 and previous config saved to /var/cache/conftool/dbconfig/20220624-053531-root.json |
[production] |
| 05:31 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1175 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30050 and previous config saved to /var/cache/conftool/dbconfig/20220624-053138-root.json |
[production] |
| 05:31 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1174 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30049 and previous config saved to /var/cache/conftool/dbconfig/20220624-053134-root.json |
[production] |
| 05:31 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1172 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30048 and previous config saved to /var/cache/conftool/dbconfig/20220624-053128-root.json |
[production] |
| 05:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1168 db1169 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30047 and previous config saved to /var/cache/conftool/dbconfig/20220624-052758-root.json |
[production] |
| 05:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1172 db1174 db1175 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30046 and previous config saved to /var/cache/conftool/dbconfig/20220624-052137-root.json |
[production] |
|
2022-06-23
§
|
| 21:23 |
<mutante> |
restbase-dev1006 has manually installed packages (wrk, maybe others) |
[production] |
| 21:22 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 21:22 |
<brennen> |
end of utc late backport & config window |
[production] |
| 21:21 |
<brennen@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:808055|[cleanup] Drop non-existent feature flags]] (duration: 03m 33s) |
[production] |
| 21:21 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 21:21 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 21:20 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 21:13 |
<thcipriani@deploy1002> |
Finished scap: Config: [[gerrit:808067|Change default skin on next set of pilot wikis to Vector (2022) (T307903)]] (duration: 17m 29s) |
[production] |
| 21:01 |
<inflatador> |
looking in to wdqs1006 alert ^^ |
[production] |
| 20:56 |
<thcipriani@deploy1002> |
Started scap: Config: [[gerrit:808067|Change default skin on next set of pilot wikis to Vector (2022) (T307903)]] |
[production] |
| 20:55 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 20:54 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 20:54 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 20:53 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 20:49 |
<thcipriani@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:808064|Enable DiscussionTools topicsubscription, autotopicsub on testwiki (T310808)]] (duration: 03m 18s) |
[production] |
| 20:48 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host dse-k8s-ctrl1001.eqiad.wmnet |
[production] |
| 20:48 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) dse-k8s-ctrl1001.eqiad.wmnet on all recursors |
[production] |
| 20:48 |
<dzahn@cumin1001> |
START - Cookbook sre.dns.wipe-cache dse-k8s-ctrl1001.eqiad.wmnet on all recursors |
[production] |
| 20:48 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) |
[production] |
| 20:48 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 20:47 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 20:47 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 20:46 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 20:43 |
<thcipriani@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:806847|ukwikibooks: Add NS102 (Рецепт) to wgContentNamespaces (T310940)]] (duration: 03m 41s) |
[production] |
| 20:43 |
<dzahn@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 20:43 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) dse-k8s-ctrl1001.eqiad.wmnet on all recursors |
[production] |
| 20:43 |
<dzahn@cumin1001> |
START - Cookbook sre.dns.wipe-cache dse-k8s-ctrl1001.eqiad.wmnet on all recursors |
[production] |
| 20:43 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) |
[production] |
| 20:41 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |