151-200 of 10000 results (16ms)
2025-09-09 ยง
18:55 <ejegg> payments-wiki upgraded from 0117408e to 10d200b1 [production]
18:52 <ejegg> donorwiki upgraded from bd6de034 to 10d200b1 [production]
18:51 <fceratto@cumin1002> dbctl commit (dc=all): 'Depooling db2164 (T402763)', diff saved to https://phabricator.wikimedia.org/P83023 and previous config saved to /var/cache/conftool/dbconfig/20250909-185158-fceratto.json [production]
18:51 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2145', diff saved to https://phabricator.wikimedia.org/P83022 and previous config saved to /var/cache/conftool/dbconfig/20250909-185146-fceratto.json [production]
18:51 <fceratto@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2164.codfw.wmnet with reason: Maintenance [production]
18:51 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2163 (T402763)', diff saved to https://phabricator.wikimedia.org/P83021 and previous config saved to /var/cache/conftool/dbconfig/20250909-185128-fceratto.json [production]
18:46 <dduvall@deploy1003> dduvall, reedy: Backport for [[gerrit:1186548|TOTP: Fix logic for displaying TOTPEnableForm (T404091 T230042)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
18:46 <bking@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
18:45 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
18:40 <dduvall@deploy1003> Started scap sync-world: Backport for [[gerrit:1186548|TOTP: Fix logic for displaying TOTPEnableForm (T404091 T230042)]] [production]
18:39 <dduvall@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.45.0-wmf.18 refs T396379 [production]
18:36 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2145', diff saved to https://phabricator.wikimedia.org/P83020 and previous config saved to /var/cache/conftool/dbconfig/20250909-183639-fceratto.json [production]
18:36 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2163', diff saved to https://phabricator.wikimedia.org/P83019 and previous config saved to /var/cache/conftool/dbconfig/20250909-183619-fceratto.json [production]
18:31 <andrew@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudcephmon2004-dev.codfw.wmnet with OS trixie [production]
18:21 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2145 (T402763)', diff saved to https://phabricator.wikimedia.org/P83018 and previous config saved to /var/cache/conftool/dbconfig/20250909-182132-fceratto.json [production]
18:21 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2163', diff saved to https://phabricator.wikimedia.org/P83017 and previous config saved to /var/cache/conftool/dbconfig/20250909-182112-fceratto.json [production]
18:14 <fceratto@cumin1002> dbctl commit (dc=all): 'Depooling db2145 (T402763)', diff saved to https://phabricator.wikimedia.org/P83016 and previous config saved to /var/cache/conftool/dbconfig/20250909-181448-fceratto.json [production]
18:14 <fceratto@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2145.codfw.wmnet with reason: Maintenance [production]
18:09 <fceratto@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2141.codfw.wmnet with reason: Maintenance [production]
18:06 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2163 (T402763)', diff saved to https://phabricator.wikimedia.org/P83015 and previous config saved to /var/cache/conftool/dbconfig/20250909-180605-fceratto.json [production]
17:58 <fceratto@cumin1002> dbctl commit (dc=all): 'Depooling db2163 (T402763)', diff saved to https://phabricator.wikimedia.org/P83014 and previous config saved to /var/cache/conftool/dbconfig/20250909-175808-fceratto.json [production]
17:58 <fceratto@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2163.codfw.wmnet with reason: Maintenance [production]
17:57 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2154 (T402763)', diff saved to https://phabricator.wikimedia.org/P83013 and previous config saved to /var/cache/conftool/dbconfig/20250909-175746-fceratto.json [production]
17:47 <andrew@cumin2002> START - Cookbook sre.hosts.reimage for host cloudcephmon2004-dev.codfw.wmnet with OS trixie [production]
17:42 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2154', diff saved to https://phabricator.wikimedia.org/P83012 and previous config saved to /var/cache/conftool/dbconfig/20250909-174239-fceratto.json [production]
17:38 <swfrench-wmf> migrated shellbox-syntaxhighlight to PHP 8.3 in eqiad - T403284 [production]
17:38 <swfrench@deploy1003> helmfile [eqiad] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
17:37 <swfrench@deploy1003> helmfile [eqiad] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
17:36 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2240 (T401906)', diff saved to https://phabricator.wikimedia.org/P83011 and previous config saved to /var/cache/conftool/dbconfig/20250909-173609-fceratto.json [production]
17:27 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2154', diff saved to https://phabricator.wikimedia.org/P83010 and previous config saved to /var/cache/conftool/dbconfig/20250909-172731-fceratto.json [production]
17:21 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2240', diff saved to https://phabricator.wikimedia.org/P83008 and previous config saved to /var/cache/conftool/dbconfig/20250909-172102-fceratto.json [production]
17:17 <wfan> payments-wiki upgraded from bd6de034 to 0117408e [production]
17:15 <swfrench-wmf> migrated shellbox-syntaxhighlight to PHP 8.3 in codfw - T403284 [production]
17:14 <swfrench@deploy1003> helmfile [codfw] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
17:14 <swfrench@deploy1003> helmfile [codfw] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
17:13 <swfrench@deploy1003> helmfile [staging] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
17:12 <swfrench@deploy1003> helmfile [staging] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
17:12 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2154 (T402763)', diff saved to https://phabricator.wikimedia.org/P83007 and previous config saved to /var/cache/conftool/dbconfig/20250909-171224-fceratto.json [production]
17:12 <swfrench-wmf> restored locally modified `helmfile.d/dse-k8s-services/_airflow_common_/values-dev.yaml` duplicating https://gerrit.wikimedia.org/r/1186497 on deploy1003 to unstick deployment-charts updates [production]
17:11 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2207 (T402763)', diff saved to https://phabricator.wikimedia.org/P83006 and previous config saved to /var/cache/conftool/dbconfig/20250909-171140-fceratto.json [production]
17:07 <fceratto@cumin1002> dbctl commit (dc=all): 'Depooling db2207 (T402763)', diff saved to https://phabricator.wikimedia.org/P83005 and previous config saved to /var/cache/conftool/dbconfig/20250909-170701-fceratto.json [production]
17:06 <fceratto@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2207.codfw.wmnet with reason: Maintenance [production]
17:05 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2240', diff saved to https://phabricator.wikimedia.org/P83004 and previous config saved to /var/cache/conftool/dbconfig/20250909-170554-fceratto.json [production]
17:04 <fceratto@cumin1002> dbctl commit (dc=all): 'Depooling db2154 (T402763)', diff saved to https://phabricator.wikimedia.org/P83003 and previous config saved to /var/cache/conftool/dbconfig/20250909-170429-fceratto.json [production]
17:04 <fceratto@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2154.codfw.wmnet with reason: Maintenance [production]
17:04 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2152 (T402763)', diff saved to https://phabricator.wikimedia.org/P83002 and previous config saved to /var/cache/conftool/dbconfig/20250909-170406-fceratto.json [production]
16:50 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2240 (T401906)', diff saved to https://phabricator.wikimedia.org/P83001 and previous config saved to /var/cache/conftool/dbconfig/20250909-165047-fceratto.json [production]
16:48 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2152', diff saved to https://phabricator.wikimedia.org/P83000 and previous config saved to /var/cache/conftool/dbconfig/20250909-164858-fceratto.json [production]
16:48 <fceratto@cumin1002> dbctl commit (dc=all): 'Depooling db2240 (T401906)', diff saved to https://phabricator.wikimedia.org/P82999 and previous config saved to /var/cache/conftool/dbconfig/20250909-164836-fceratto.json [production]
16:48 <fceratto@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2240.codfw.wmnet with reason: Maintenance [production]