|
2026-04-17
§
|
| 15:18 |
<otto@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply |
[production] |
| 15:17 |
<otto@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply |
[production] |
| 15:17 |
<otto@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply |
[production] |
| 15:17 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Depooling db2208 (T419635)', diff saved to https://phabricator.wikimedia.org/P91088 and previous config saved to /var/cache/conftool/dbconfig/20260417-151723-fceratto.json |
[production] |
| 15:17 |
<fceratto@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2208.codfw.wmnet with reason: Maintenance |
[production] |
| 15:16 |
<fceratto@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2200.codfw.wmnet with reason: Maintenance |
[production] |
| 15:15 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1254', diff saved to https://phabricator.wikimedia.org/P91087 and previous config saved to /var/cache/conftool/dbconfig/20260417-151541-fceratto.json |
[production] |
| 15:05 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1254', diff saved to https://phabricator.wikimedia.org/P91086 and previous config saved to /var/cache/conftool/dbconfig/20260417-150532-fceratto.json |
[production] |
| 15:04 |
<fceratto@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2198.codfw.wmnet with reason: Maintenance |
[production] |
| 15:04 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2182 (T419635)', diff saved to https://phabricator.wikimedia.org/P91085 and previous config saved to /var/cache/conftool/dbconfig/20260417-150440-fceratto.json |
[production] |
| 14:55 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1254 (T419961)', diff saved to https://phabricator.wikimedia.org/P91084 and previous config saved to /var/cache/conftool/dbconfig/20260417-145524-fceratto.json |
[production] |
| 14:54 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2182', diff saved to https://phabricator.wikimedia.org/P91083 and previous config saved to /var/cache/conftool/dbconfig/20260417-145432-fceratto.json |
[production] |
| 14:51 |
<otto@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply |
[production] |
| 14:51 |
<otto@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply |
[production] |
| 14:48 |
<otto@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply |
[production] |
| 14:48 |
<otto@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply |
[production] |
| 14:48 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Depooling db1254 (T419961)', diff saved to https://phabricator.wikimedia.org/P91082 and previous config saved to /var/cache/conftool/dbconfig/20260417-144819-fceratto.json |
[production] |
| 14:48 |
<fceratto@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1254.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:44 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2182', diff saved to https://phabricator.wikimedia.org/P91081 and previous config saved to /var/cache/conftool/dbconfig/20260417-144424-fceratto.json |
[production] |
| 14:43 |
<fceratto@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1239.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:42 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1233 (T419961)', diff saved to https://phabricator.wikimedia.org/P91080 and previous config saved to /var/cache/conftool/dbconfig/20260417-144247-fceratto.json |
[production] |
| 14:36 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'edit-check' for release 'main' . |
[production] |
| 14:36 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 14:34 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2182 (T419635)', diff saved to https://phabricator.wikimedia.org/P91079 and previous config saved to /var/cache/conftool/dbconfig/20260417-143416-fceratto.json |
[production] |
| 14:33 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 14:33 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 14:33 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 14:32 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 14:32 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 14:32 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1233', diff saved to https://phabricator.wikimedia.org/P91078 and previous config saved to /var/cache/conftool/dbconfig/20260417-143238-fceratto.json |
[production] |
| 14:32 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Depooling db2182 (T419635)', diff saved to https://phabricator.wikimedia.org/P91077 and previous config saved to /var/cache/conftool/dbconfig/20260417-143204-fceratto.json |
[production] |
| 14:31 |
<fceratto@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2182.codfw.wmnet with reason: Maintenance |
[production] |
| 14:31 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2168 (T419635)', diff saved to https://phabricator.wikimedia.org/P91076 and previous config saved to /var/cache/conftool/dbconfig/20260417-143139-fceratto.json |
[production] |
| 14:22 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1233', diff saved to https://phabricator.wikimedia.org/P91075 and previous config saved to /var/cache/conftool/dbconfig/20260417-142230-fceratto.json |
[production] |
| 14:21 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2168', diff saved to https://phabricator.wikimedia.org/P91074 and previous config saved to /var/cache/conftool/dbconfig/20260417-142130-fceratto.json |
[production] |
| 14:18 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 14:16 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 14:16 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 14:14 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 14:13 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 14:12 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 14:12 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 14:12 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1233 (T419961)', diff saved to https://phabricator.wikimedia.org/P91073 and previous config saved to /var/cache/conftool/dbconfig/20260417-141222-fceratto.json |
[production] |
| 14:12 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 14:11 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 14:11 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2168', diff saved to https://phabricator.wikimedia.org/P91072 and previous config saved to /var/cache/conftool/dbconfig/20260417-141123-fceratto.json |
[production] |
| 14:10 |
<dpogorzelski@deploy1003> |
helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 14:09 |
<urandom> |
decommissioning Cassandra, aqs1011 [a,b] — T412830 |
[production] |
| 14:06 |
<fabfur@cumin1003> |
END (PASS) - Cookbook sre.cdn.roll-restart-varnish (exit_code=0) rolling restart of Varnish on 1 hosts matching query P{cp3073.*} |
[production] |
| 14:06 |
<eevans@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 30 days, 0:00:00 on aqs1011.eqiad.wmnet with reason: Bootstrapping — T412830 |
[production] |