501-550 of 10000 results (26ms)
2026-02-17 ยง
13:11 <ayounsi@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cr2-magru,cr2-magru IPv6 with reason: router upgrade [production]
13:08 <cgoubert@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1030.eqiad.wmnet [production]
13:08 <cgoubert@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1030.eqiad.wmnet [production]
13:05 <cgoubert@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1030.eqiad.wmnet [production]
13:04 <cgoubert@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1030.eqiad.wmnet [production]
13:03 <ayounsi@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cr1-magru,cr1-magru IPv6 with reason: router upgrade [production]
13:02 <ayounsi@cumin1003> DONE (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on cr1-magru,cr1-magru IPv6,cr1-magru.mgmt with reason: router upgrade [production]
13:01 <ayounsi@cumin1003> conftool action : set/pooled=no; selector: cluster=dnsbox,dc=magru [reason: magru maintenance] [production]
13:00 <ayounsi@cumin1003> END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: depool site magru [reason: no reason specified, T416442] [production]
13:00 <ayounsi@cumin1003> START - Cookbook sre.dns.admin DNS admin: depool site magru [reason: no reason specified, T416442] [production]
12:40 <reedy@deploy2002> Finished scap sync-world: Backport for [[gerrit:1233679|CommonSettings.php: Stop loading WebAuthn (T303495)]], [[gerrit:1233680|wmf-config: Remove $wmgUseWebAuthn and extension from extension-list (T303495)]] (duration: 10m 58s) [production]
12:34 <reedy@deploy2002> reedy: Continuing with sync [production]
12:33 <reedy@deploy2002> reedy: Backport for [[gerrit:1233679|CommonSettings.php: Stop loading WebAuthn (T303495)]], [[gerrit:1233680|wmf-config: Remove $wmgUseWebAuthn and extension from extension-list (T303495)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
12:29 <reedy@deploy2002> Started scap sync-world: Backport for [[gerrit:1233679|CommonSettings.php: Stop loading WebAuthn (T303495)]], [[gerrit:1233680|wmf-config: Remove $wmgUseWebAuthn and extension from extension-list (T303495)]] [production]
12:28 <volans> re-enabled puppet on toolforge's nfs k8s workers [admin]
12:15 <jiji@deploy2002> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
12:05 <jiji@deploy2002> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
11:59 <jiji@deploy2002> helmfile [eqiad] DONE helmfile.d/admin 'apply'. [production]
11:55 <jiji@deploy2002> helmfile [eqiad] START helmfile.d/admin 'apply'. [production]
11:50 <jiji@deploy2002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
11:46 <jiji@deploy2002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
11:42 <jiji@deploy2002> helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. [production]
11:40 <jiji@deploy2002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
11:29 <dreamyjazz@deploy2002> Finished scap sync-world: Backport for [[gerrit:1239901|Take locked users into account for case auto-closure (T417013)]] (duration: 37m 25s) [production]
11:17 <dreamyjazz@deploy2002> dreamyjazz: Continuing with sync [production]
11:16 <dreamyjazz@deploy2002> dreamyjazz: Backport for [[gerrit:1239901|Take locked users into account for case auto-closure (T417013)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
11:16 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db1190 (T415786)', diff saved to https://phabricator.wikimedia.org/P88840 and previous config saved to /var/cache/conftool/dbconfig/20260217-111643-marostegui.json [production]
11:16 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1190.eqiad.wmnet with reason: Maintenance [production]
11:16 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1160 (T415786)', diff saved to https://phabricator.wikimedia.org/P88839 and previous config saved to /var/cache/conftool/dbconfig/20260217-111618-marostegui.json [production]
11:06 <moritzm> upgrading clamav on vrts1003 to 1.4.3 [production]
11:01 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1160', diff saved to https://phabricator.wikimedia.org/P88838 and previous config saved to /var/cache/conftool/dbconfig/20260217-110109-marostegui.json [production]
10:52 <dreamyjazz@deploy2002> Started scap sync-world: Backport for [[gerrit:1239901|Take locked users into account for case auto-closure (T417013)]] [production]
10:46 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1160', diff saved to https://phabricator.wikimedia.org/P88837 and previous config saved to /var/cache/conftool/dbconfig/20260217-104601-marostegui.json [production]
10:44 <moritzm> upgrading clamav on vrts2002 to 1.4.3 [production]
10:39 <volans> temporarily disabling puppet on toolforge's nfs k8s workers to test gerrit/1239689 [admin]
10:30 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1160 (T415786)', diff saved to https://phabricator.wikimedia.org/P88836 and previous config saved to /var/cache/conftool/dbconfig/20260217-103053-marostegui.json [production]
10:26 <kevinbazira@deploy2002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' . [production]
10:24 <kevinbazira@deploy2002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' . [production]
09:44 <jayme@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host kubestage2001.codfw.wmnet [production]
09:44 <jayme@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host kubestage2001.codfw.wmnet [production]
09:34 <trueg@deploy2002> helmfile [eqiad] DONE helmfile.d/services/rdf-streaming-updater: apply [production]
09:33 <trueg@deploy2002> helmfile [eqiad] START helmfile.d/services/rdf-streaming-updater: apply [production]
09:26 <trueg@deploy2002> helmfile [codfw] DONE helmfile.d/services/rdf-streaming-updater: apply [production]
09:25 <trueg@deploy2002> helmfile [codfw] START helmfile.d/services/rdf-streaming-updater: apply [production]
09:24 <trueg@deploy2002> helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply [production]
09:24 <trueg@deploy2002> helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply [production]
09:18 <jayme@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kubestage2001.codfw.wmnet with OS trixie [production]
09:01 <hashar> Reloaded Zuul to enable php 8.5 testing on utfnormal, php-session-serializer, wikipeg, mediawiki/libs/Dodo, mediawiki/libs/UUID, testing-access-wrapper and translatewiki # T406326 [releng]
08:58 <jayme@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kubestage2001.codfw.wmnet with reason: host reimage [production]
08:56 <trueg@deploy2002> helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply [production]