1951-2000 of 10000 results (38ms)
2022-02-09 ยง
13:17 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
12:46 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
12:42 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
12:42 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
12:41 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
12:41 <Lucas_WMDE> UTC morning backport+config window done [production]
12:40 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:761310|sawikisource: Add audio book namespace (T282970)]] (duration: 00m 50s) [production]
12:21 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
12:15 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
12:15 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
12:14 <lucaswerkmeister-wmde@deploy1002> Synchronized multiversion/MWRealm.php: Config: [[gerrit:760640|Stop writing to $wmfRealm (T45956)]] (3/3) (duration: 00m 49s) [production]
12:13 <lucaswerkmeister-wmde@deploy1002> Synchronized multiversion/buildConfigCache.php: Config: [[gerrit:760640|Stop writing to $wmfRealm (T45956)]] (2/3) (duration: 00m 49s) [production]
12:11 <lucaswerkmeister-wmde@deploy1002> Synchronized tests/loggingTest.php: Config: [[gerrit:760640|Stop writing to $wmfRealm (T45956)]] (1/3) (duration: 01m 38s) [production]
12:10 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
11:20 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1100 (T300775)', diff saved to https://phabricator.wikimedia.org/P20411 and previous config saved to /var/cache/conftool/dbconfig/20220209-112029-marostegui.json [production]
11:20 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1100.eqiad.wmnet with reason: Maintenance [production]
11:20 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db1100.eqiad.wmnet with reason: Maintenance [production]
11:08 <mvernon@cumin2002> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts ms-fe[2005-2008].codfw.wmnet [production]
10:50 <mvernon@cumin2002> START - Cookbook sre.hosts.decommission for hosts ms-fe[2005-2008].codfw.wmnet [production]
10:45 <akosiaris> T300568 upload prometheus-etherpad-exporter_0.5_amd64 to apt.wikimedia.org bullseye-wikimedia/main [production]
10:35 <jayme@deploy1002> helmfile [staging] DONE helmfile.d/services/miscweb: sync on main [production]
10:34 <jayme@deploy1002> helmfile [staging] START helmfile.d/services/miscweb: apply on main [production]
10:34 <jayme@deploy1002> helmfile [staging] DONE helmfile.d/services/miscweb: sync on main [production]
10:32 <jayme@deploy1002> helmfile [staging] START helmfile.d/services/miscweb: apply on main [production]
10:25 <jelto@deploy1002> Finished deploy [restbase/deploy@0848b15] (dev-cluster): (no justification provided) (duration: 00m 22s) [production]
10:25 <jelto@deploy1002> Started deploy [restbase/deploy@0848b15] (dev-cluster): (no justification provided) [production]
10:20 <jelto> update scap to 4.3.1 on A:restbase-canary - T301307 [production]
10:17 <jelto> update scap to 4.3.1 on A:mw-canary or A:parsoid-canary or A:mw-jobrunner-canary - T301307 [production]
10:16 <ariel@deploy1002> Finished deploy [dumps/dumps@9993036]: fix up default api jobs entry for siteinfo v2 (duration: 00m 03s) [production]
10:15 <ariel@deploy1002> Started deploy [dumps/dumps@9993036]: fix up default api jobs entry for siteinfo v2 [production]
10:15 <mvernon@cumin2002> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=99) for hosts ms-fe[2005-2008].codfw.wmnet [production]
10:14 <volans> uploaded python3-wmflib_1.0.1 to apt.wikimedia.org buster-wikimedia,bullseye-wikimedia [production]
10:11 <mvernon@cumin2002> START - Cookbook sre.hosts.decommission for hosts ms-fe[2005-2008].codfw.wmnet [production]
10:03 <akosiaris> T300568 upload prometheus-etherpad-exporter_0.4_amd64 to apt.wikimedia.org bullseye-wikimedia/main [production]
10:02 <Emperor> rolling restart of swift frontends T301251 [production]
09:46 <jayme@deploy1002> helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. [production]
09:45 <jayme@deploy1002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
09:45 <jayme@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
09:45 <elukey> update my ssh key on all network devices (will commit only when the diff is my key only) [production]
09:44 <jayme@deploy1002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
09:41 <ema> cp3050: stop and disable atskafka-webrequest.service T247497 [production]
09:15 <ema> cp3050: ats-backend-restart to set the number of allowed Lua states back from 64 to 256 (default) T265625 [production]
08:21 <dcausse> restarting blazegraph on wdqs1004 (jvm stuck for 5hours) [production]
07:55 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be2001.codfw.wmnet [production]
07:42 <filippo@cumin1001> START - Cookbook sre.hosts.reboot-single for host thanos-be2001.codfw.wmnet [production]
07:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Remove logpager group from s1 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P20410 and previous config saved to /var/cache/conftool/dbconfig/20220209-073528-marostegui.json [production]
04:10 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1145.eqiad.wmnet with reason: Maintenance [production]
04:10 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1145.eqiad.wmnet with reason: Maintenance [production]
03:48 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
03:48 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]