1-50 of 10000 results (97ms)
2026-01-26 ยง
16:57 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) db1224 gradually with 4 steps - Pool db1224.eqiad.wmnet in after cloning [production]
16:50 <jgiannelos@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mobileapps: apply [production]
16:49 <jgiannelos@deploy2002> helmfile [eqiad] START helmfile.d/services/mobileapps: apply [production]
16:48 <jgiannelos@deploy2002> helmfile [codfw] DONE helmfile.d/services/mobileapps: apply [production]
16:47 <jgiannelos@deploy2002> helmfile [codfw] START helmfile.d/services/mobileapps: apply [production]
16:47 <jdrewniak@deploy2002> Finished scap sync-world: Backport for [[gerrit:1230961|Bumping portals submodule to master (T128546)]] (duration: 08m 57s) [production]
16:46 <jgiannelos@deploy2002> helmfile [staging] DONE helmfile.d/services/mobileapps: apply [production]
16:46 <jgiannelos@deploy2002> helmfile [staging] START helmfile.d/services/mobileapps: apply [production]
16:44 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.sanitize-wiki (exit_code=0) Checking sanitization for wikis kajwiki in section s5 [production]
16:43 <jdrewniak@deploy2002> jdrewniak: Continuing with sync [production]
16:41 <marostegui@cumin1003> START - Cookbook sre.mysql.sanitize-wiki Checking sanitization for wikis kajwiki in section s5 [production]
16:41 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.sanitize-wiki (exit_code=0) Managing sanitization for wikis kajwiki in section s5 [production]
16:40 <jdrewniak@deploy2002> jdrewniak: Backport for [[gerrit:1230961|Bumping portals submodule to master (T128546)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
16:38 <jdrewniak@deploy2002> Started scap sync-world: Backport for [[gerrit:1230961|Bumping portals submodule to master (T128546)]] [production]
16:35 <urbanecm@deploy2002> Finished scap sync-world: Backport for [[gerrit:1233210|kajwiki: Update logos and timezone (T413283)]] (duration: 06m 32s) [production]
16:34 <marostegui@cumin1003> START - Cookbook sre.mysql.sanitize-wiki Managing sanitization for wikis kajwiki in section s5 [production]
16:31 <urbanecm@deploy2002> urbanecm: Continuing with sync [production]
16:31 <urbanecm@deploy2002> urbanecm: Backport for [[gerrit:1233210|kajwiki: Update logos and timezone (T413283)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
16:29 <urbanecm@deploy2002> Started scap sync-world: Backport for [[gerrit:1233210|kajwiki: Update logos and timezone (T413283)]] [production]
16:19 <dancy@deploy2002> Installation of scap version "4.236.0" completed for 2 hosts [production]
16:17 <dancy@deploy2002> Installing scap version "4.236.0" for 2 host(s) [production]
16:12 <marostegui@cumin1003> START - Cookbook sre.mysql.pool db1224 gradually with 4 steps - Pool db1224.eqiad.wmnet in after cloning [production]
16:06 <urbanecm@deploy2002> Finished scap sync-world: Backport for [[gerrit:1233206|kajwiki: Enable GrowthExperiments (T415027)]] (duration: 10m 04s) [production]
16:01 <urbanecm@deploy2002> urbanecm: Continuing with sync [production]
15:57 <urbanecm@deploy2002> urbanecm: Backport for [[gerrit:1233206|kajwiki: Enable GrowthExperiments (T415027)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
15:56 <urbanecm@deploy2002> Started scap sync-world: Backport for [[gerrit:1233206|kajwiki: Enable GrowthExperiments (T415027)]] [production]
15:49 <urbanecm@deploy2002> Finished scap sync-world: Backport for [[gerrit:1233205|Activate kajwiki (T413283)]] (duration: 07m 03s) [production]
15:45 <urbanecm@deploy2002> urbanecm: Continuing with sync [production]
15:44 <urbanecm@deploy2002> urbanecm: Backport for [[gerrit:1233205|Activate kajwiki (T413283)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
15:42 <urbanecm@deploy2002> Started scap sync-world: Backport for [[gerrit:1233205|Activate kajwiki (T413283)]] [production]
15:25 <urbanecm@deploy2002> Finished scap sync-world: Backport for [[gerrit:1233204|Prepare configuration for kajwiki (T413283)]] (duration: 07m 54s) [production]
15:24 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db2181 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P87932 and previous config saved to /var/cache/conftool/dbconfig/20260126-152436-marostegui.json [production]
15:24 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db2181.codfw.wmnet with reason: Maintenance [production]
15:24 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2167 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P87931 and previous config saved to /var/cache/conftool/dbconfig/20260126-152411-marostegui.json [production]
15:21 <urbanecm@deploy2002> urbanecm: Continuing with sync [production]
15:19 <urbanecm@deploy2002> urbanecm: Backport for [[gerrit:1233204|Prepare configuration for kajwiki (T413283)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
15:17 <urbanecm@deploy2002> Started scap sync-world: Backport for [[gerrit:1233204|Prepare configuration for kajwiki (T413283)]] [production]
15:14 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2167', diff saved to https://phabricator.wikimedia.org/P87930 and previous config saved to /var/cache/conftool/dbconfig/20260126-151403-marostegui.json [production]
15:03 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2167', diff saved to https://phabricator.wikimedia.org/P87929 and previous config saved to /var/cache/conftool/dbconfig/20260126-150355-marostegui.json [production]
14:53 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2167 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P87928 and previous config saved to /var/cache/conftool/dbconfig/20260126-145346-marostegui.json [production]
14:50 <Lucas_WMDE> UTC afternoon backport+config window done [production]
14:49 <lucaswerkmeister-wmde@deploy2002> Finished scap sync-world: Backport for [[gerrit:1225649|Defensively set Abstract Wikipedia feature flags to false (T411690 T411691)]] (duration: 07m 24s) [production]
14:45 <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde, jforrester: Continuing with sync [production]
14:44 <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde, jforrester: Backport for [[gerrit:1225649|Defensively set Abstract Wikipedia feature flags to false (T411690 T411691)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
14:42 <lucaswerkmeister-wmde@deploy2002> Started scap sync-world: Backport for [[gerrit:1225649|Defensively set Abstract Wikipedia feature flags to false (T411690 T411691)]] [production]
14:39 <sgimeno@deploy2002> Finished scap sync-world: Backport for [[gerrit:1233116|fix: avoid logging traffic from overridden experiment users (T415294)]], [[gerrit:1233117|fix: avoid displaying incorrect additional userpage link (T415291)]] (duration: 06m 54s) [production]
14:35 <sgimeno@deploy2002> sgimeno: Continuing with sync [production]
14:34 <sgimeno@deploy2002> sgimeno: Backport for [[gerrit:1233116|fix: avoid logging traffic from overridden experiment users (T415294)]], [[gerrit:1233117|fix: avoid displaying incorrect additional userpage link (T415291)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
14:32 <sgimeno@deploy2002> Started scap sync-world: Backport for [[gerrit:1233116|fix: avoid logging traffic from overridden experiment users (T415294)]], [[gerrit:1233117|fix: avoid displaying incorrect additional userpage link (T415291)]] [production]
14:16 <lucaswerkmeister-wmde@deploy2002> Finished scap sync-world: Backport for [[gerrit:1230498|rowiki: Set noindex for User: and User talk: Namespaces (T414992)]] (duration: 12m 36s) [production]