2201-2250 of 10000 results (47ms)
2022-02-09 ยง
13:55 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2126.codfw.wmnet with reason: Maintenance [production]
13:54 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2095.codfw.wmnet with reason: Migrate to bullseye (T300510) [production]
13:53 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2095.codfw.wmnet with reason: Migrate to bullseye (T300510) [production]
13:48 <jelto> update scap to 4.3.1 on all hosts - T301307 [production]
13:38 <reedy@deploy1002> Finished scap: Downgrading symfony/console \(v5.4.3 => v5.4.2\) T301320 (duration: 01m 34s) [production]
13:36 <reedy@deploy1002> Started scap: Downgrading symfony/console \(v5.4.3 => v5.4.2\) T301320 [production]
13:19 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1164 (T298554)', diff saved to https://phabricator.wikimedia.org/P20412 and previous config saved to /var/cache/conftool/dbconfig/20220209-131938-ladsgroup.json [production]
13:19 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1164.eqiad.wmnet with reason: Maintenance [production]
13:19 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1164.eqiad.wmnet with reason: Maintenance [production]
13:19 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
13:18 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
13:18 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
13:17 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
12:46 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
12:42 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
12:42 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
12:41 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
12:41 <Lucas_WMDE> UTC morning backport+config window done [production]
12:40 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:761310|sawikisource: Add audio book namespace (T282970)]] (duration: 00m 50s) [production]
12:21 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
12:15 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
12:15 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
12:14 <lucaswerkmeister-wmde@deploy1002> Synchronized multiversion/MWRealm.php: Config: [[gerrit:760640|Stop writing to $wmfRealm (T45956)]] (3/3) (duration: 00m 49s) [production]
12:13 <lucaswerkmeister-wmde@deploy1002> Synchronized multiversion/buildConfigCache.php: Config: [[gerrit:760640|Stop writing to $wmfRealm (T45956)]] (2/3) (duration: 00m 49s) [production]
12:11 <lucaswerkmeister-wmde@deploy1002> Synchronized tests/loggingTest.php: Config: [[gerrit:760640|Stop writing to $wmfRealm (T45956)]] (1/3) (duration: 01m 38s) [production]
12:10 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
11:20 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1100 (T300775)', diff saved to https://phabricator.wikimedia.org/P20411 and previous config saved to /var/cache/conftool/dbconfig/20220209-112029-marostegui.json [production]
11:20 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1100.eqiad.wmnet with reason: Maintenance [production]
11:20 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db1100.eqiad.wmnet with reason: Maintenance [production]
11:08 <mvernon@cumin2002> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts ms-fe[2005-2008].codfw.wmnet [production]
10:50 <mvernon@cumin2002> START - Cookbook sre.hosts.decommission for hosts ms-fe[2005-2008].codfw.wmnet [production]
10:45 <akosiaris> T300568 upload prometheus-etherpad-exporter_0.5_amd64 to apt.wikimedia.org bullseye-wikimedia/main [production]
10:35 <jayme@deploy1002> helmfile [staging] DONE helmfile.d/services/miscweb: sync on main [production]
10:34 <jayme@deploy1002> helmfile [staging] START helmfile.d/services/miscweb: apply on main [production]
10:34 <jayme@deploy1002> helmfile [staging] DONE helmfile.d/services/miscweb: sync on main [production]
10:32 <jayme@deploy1002> helmfile [staging] START helmfile.d/services/miscweb: apply on main [production]
10:25 <jelto@deploy1002> Finished deploy [restbase/deploy@0848b15] (dev-cluster): (no justification provided) (duration: 00m 22s) [production]
10:25 <jelto@deploy1002> Started deploy [restbase/deploy@0848b15] (dev-cluster): (no justification provided) [production]
10:20 <jelto> update scap to 4.3.1 on A:restbase-canary - T301307 [production]
10:17 <jelto> update scap to 4.3.1 on A:mw-canary or A:parsoid-canary or A:mw-jobrunner-canary - T301307 [production]
10:16 <ariel@deploy1002> Finished deploy [dumps/dumps@9993036]: fix up default api jobs entry for siteinfo v2 (duration: 00m 03s) [production]
10:15 <ariel@deploy1002> Started deploy [dumps/dumps@9993036]: fix up default api jobs entry for siteinfo v2 [production]
10:15 <mvernon@cumin2002> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=99) for hosts ms-fe[2005-2008].codfw.wmnet [production]
10:14 <volans> uploaded python3-wmflib_1.0.1 to apt.wikimedia.org buster-wikimedia,bullseye-wikimedia [production]
10:11 <mvernon@cumin2002> START - Cookbook sre.hosts.decommission for hosts ms-fe[2005-2008].codfw.wmnet [production]
10:03 <akosiaris> T300568 upload prometheus-etherpad-exporter_0.4_amd64 to apt.wikimedia.org bullseye-wikimedia/main [production]
10:02 <Emperor> rolling restart of swift frontends T301251 [production]
09:46 <jayme@deploy1002> helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. [production]
09:45 <jayme@deploy1002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
09:45 <jayme@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]