1201-1250 of 10000 results (86ms)
2024-06-26 ยง
11:54 <mvolz@deploy1002> helmfile [codfw] START helmfile.d/services/citoid: apply [production]
11:54 <mvolz@deploy1002> helmfile [staging] DONE helmfile.d/services/citoid: apply [production]
11:54 <mvolz@deploy1002> helmfile [staging] START helmfile.d/services/citoid: apply [production]
11:52 <mvolz@deploy1002> helmfile [staging] DONE helmfile.d/services/citoid: apply [production]
11:52 <mvolz@deploy1002> helmfile [staging] START helmfile.d/services/citoid: apply [production]
11:51 <mvolz@deploy1002> helmfile [staging] DONE helmfile.d/services/citoid: apply [production]
11:51 <mvolz@deploy1002> helmfile [staging] START helmfile.d/services/citoid: apply [production]
11:44 <mvolz@deploy1002> helmfile [eqiad] DONE helmfile.d/services/citoid: apply [production]
11:43 <mvolz@deploy1002> helmfile [eqiad] START helmfile.d/services/citoid: apply [production]
11:41 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2138', diff saved to https://phabricator.wikimedia.org/P65468 and previous config saved to /var/cache/conftool/dbconfig/20240626-114121-marostegui.json [production]
11:41 <mvolz@deploy1002> helmfile [codfw] DONE helmfile.d/services/citoid: apply [production]
11:40 <mvolz@deploy1002> helmfile [codfw] START helmfile.d/services/citoid: apply [production]
11:39 <mvolz@deploy1002> helmfile [staging] DONE helmfile.d/services/citoid: apply [production]
11:39 <mvolz@deploy1002> helmfile [staging] START helmfile.d/services/citoid: apply [production]
11:35 <moritzm> installing emacs security updates [production]
11:27 <mvolz@deploy1002> helmfile [codfw] DONE helmfile.d/services/zotero: apply [production]
11:26 <mvolz@deploy1002> helmfile [codfw] START helmfile.d/services/zotero: apply [production]
11:26 <mvolz@deploy1002> helmfile [staging] DONE helmfile.d/services/zotero: apply [production]
11:26 <mvolz@deploy1002> helmfile [staging] START helmfile.d/services/zotero: apply [production]
11:26 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2138 (T367856)', diff saved to https://phabricator.wikimedia.org/P65467 and previous config saved to /var/cache/conftool/dbconfig/20240626-112614-marostegui.json [production]
11:24 <root@cumin1002> START - Cookbook sre.hosts.reimage for host cloudcephosd1006.eqiad.wmnet with OS bullseye [production]
11:24 <mvolz@deploy1002> helmfile [staging] DONE helmfile.d/services/zotero: apply [production]
11:23 <mvolz@deploy1002> helmfile [staging] START helmfile.d/services/zotero: apply [production]
11:19 <jynus@cumin1002> dbctl commit (dc=all): 'Repool es2022 fully T363812', diff saved to https://phabricator.wikimedia.org/P65466 and previous config saved to /var/cache/conftool/dbconfig/20240626-111934-jynus.json [production]
11:14 <mvolz@deploy1002> helmfile [codfw] DONE helmfile.d/services/zotero: apply [production]
11:13 <mvolz@deploy1002> helmfile [codfw] START helmfile.d/services/zotero: apply [production]
11:12 <mvolz@deploy1002> helmfile [staging] DONE helmfile.d/services/zotero: apply [production]
11:12 <mvolz@deploy1002> helmfile [staging] START helmfile.d/services/zotero: apply [production]
11:07 <mvolz@deploy1002> helmfile [codfw] DONE helmfile.d/services/zotero: apply [production]
11:07 <mvolz@deploy1002> helmfile [codfw] START helmfile.d/services/zotero: apply [production]
11:06 <mvolz@deploy1002> helmfile [staging] DONE helmfile.d/services/zotero: apply [production]
11:06 <mvolz@deploy1002> helmfile [staging] START helmfile.d/services/zotero: apply [production]
10:39 <jynus@cumin1002> dbctl commit (dc=all): 'Repool es2022 at 50% T363812', diff saved to https://phabricator.wikimedia.org/P65465 and previous config saved to /var/cache/conftool/dbconfig/20240626-103933-jynus.json [production]
10:25 <jynus@cumin1002> dbctl commit (dc=all): 'Repool es2022 after backup T363812', diff saved to https://phabricator.wikimedia.org/P65464 and previous config saved to /var/cache/conftool/dbconfig/20240626-102523-jynus.json [production]
10:20 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . [production]
10:13 <claime> enabling puppet on cp-text - T367949 [production]
10:04 <claime> enabling puppet on cp4037 - T367949 [production]
10:02 <claime> disabling puppet on cp-text - T367949 [production]
09:58 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
09:55 <slyngs> Update idp.wikimedia.org to CAS 6.6.15.2 (T368503) [production]
09:50 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' . [production]
09:48 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'readability' for release 'main' . [production]
09:46 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'llm' for release 'main' . [production]
09:44 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . [production]
09:38 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'article-descriptions' for release 'main' . [production]
09:01 <slyngshede@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host idp-test1002.wikimedia.org with OS bookworm [production]
08:55 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2136 T365805', diff saved to https://phabricator.wikimedia.org/P65463 and previous config saved to /var/cache/conftool/dbconfig/20240626-085511-root.json [production]
08:44 <elukey@cumin1002> END (PASS) - Cookbook sre.puppet.renew-cert (exit_code=0) for puppetmaster1003.eqiad.wmnet: Renew puppet certificate - elukey@cumin1002 [production]
08:42 <elukey@cumin1002> START - Cookbook sre.puppet.renew-cert for puppetmaster1003.eqiad.wmnet: Renew puppet certificate - elukey@cumin1002 [production]
08:40 <slyngshede@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on idp-test1002.wikimedia.org with reason: host reimage [production]