3751-3800 of 10000 results (83ms)
2022-10-26 ยง
08:25 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on ganeti1015.eqiad.wmnet with reason: Remove from cluster for eventual reimage [production]
08:25 <jmm@cumin2002> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on ganeti1015.eqiad.wmnet with reason: Remove from cluster for eventual reimage [production]
08:20 <oblivian@deploy1002> helmfile [codfw] DONE helmfile.d/services/shellbox-timeline: apply [production]
08:19 <oblivian@deploy1002> helmfile [codfw] START helmfile.d/services/shellbox-timeline: apply [production]
08:18 <oblivian@deploy1002> helmfile [eqiad] DONE helmfile.d/services/shellbox-timeline: apply [production]
08:18 <oblivian@deploy1002> helmfile [eqiad] START helmfile.d/services/shellbox-timeline: apply [production]
08:18 <oblivian@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-timeline: apply [production]
08:17 <oblivian@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-timeline: apply [production]
08:16 <marostegui@cumin1001> dbctl commit (dc=all): 'es2029 (re)pooling @ 75%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36415 and previous config saved to /var/cache/conftool/dbconfig/20221026-081652-root.json [production]
08:16 <marostegui@cumin1001> dbctl commit (dc=all): 'es2030 (re)pooling @ 75%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36414 and previous config saved to /var/cache/conftool/dbconfig/20221026-081644-root.json [production]
08:16 <marostegui@cumin1001> dbctl commit (dc=all): 'es2031 (re)pooling @ 75%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36413 and previous config saved to /var/cache/conftool/dbconfig/20221026-081637-root.json [production]
08:16 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
08:15 <oblivian@deploy1002> helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply [production]
08:14 <oblivian@deploy1002> helmfile [eqiad] START helmfile.d/services/shellbox-media: apply [production]
08:12 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
08:12 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
08:12 <jnuche@deploy1002> Synchronized php: group1 wikis to 1.40.0-wmf.7 refs T320512 (duration: 03m 46s) [production]
08:12 <oblivian@deploy1002> helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply [production]
08:11 <oblivian@deploy1002> helmfile [codfw] START helmfile.d/services/shellbox-media: apply [production]
08:11 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
08:11 <oblivian@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-media: apply [production]
08:10 <oblivian@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-media: apply [production]
08:08 <jnuche@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.40.0-wmf.7 refs T320512 [production]
08:06 <marostegui@cumin1001> dbctl commit (dc=all): 'es1031 (re)pooling @ 100%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36412 and previous config saved to /var/cache/conftool/dbconfig/20221026-080643-root.json [production]
08:06 <marostegui@cumin1001> dbctl commit (dc=all): 'es1030 (re)pooling @ 100%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36411 and previous config saved to /var/cache/conftool/dbconfig/20221026-080638-root.json [production]
08:06 <marostegui@cumin1001> dbctl commit (dc=all): 'es1029 (re)pooling @ 100%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36410 and previous config saved to /var/cache/conftool/dbconfig/20221026-080631-root.json [production]
08:02 <oblivian@deploy1002> helmfile [eqiad] DONE helmfile.d/services/shellbox-constraints: apply [production]
08:02 <oblivian@deploy1002> helmfile [eqiad] START helmfile.d/services/shellbox-constraints: apply [production]
08:01 <marostegui@cumin1001> dbctl commit (dc=all): 'es2029 (re)pooling @ 50%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36409 and previous config saved to /var/cache/conftool/dbconfig/20221026-080147-root.json [production]
08:01 <marostegui@cumin1001> dbctl commit (dc=all): 'es2030 (re)pooling @ 50%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36408 and previous config saved to /var/cache/conftool/dbconfig/20221026-080139-root.json [production]
08:01 <marostegui@cumin1001> dbctl commit (dc=all): 'es2031 (re)pooling @ 50%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36407 and previous config saved to /var/cache/conftool/dbconfig/20221026-080132-root.json [production]
08:00 <oblivian@deploy1002> helmfile [codfw] DONE helmfile.d/services/shellbox-constraints: apply [production]
08:00 <oblivian@deploy1002> helmfile [codfw] START helmfile.d/services/shellbox-constraints: apply [production]
08:00 <jbond> upload python3.9 packages for buster (component python39) [production]
07:57 <oblivian@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-constraints: apply [production]
07:57 <oblivian@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-constraints: apply [production]
07:56 <oblivian@deploy1002> helmfile [eqiad] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
07:56 <oblivian@deploy1002> helmfile [eqiad] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
07:55 <oblivian@deploy1002> helmfile [codfw] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
07:55 <oblivian@deploy1002> helmfile [codfw] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
07:53 <oblivian@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
07:52 <oblivian@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
07:51 <marostegui@cumin1001> dbctl commit (dc=all): 'es1031 (re)pooling @ 75%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36406 and previous config saved to /var/cache/conftool/dbconfig/20221026-075138-root.json [production]
07:51 <marostegui@cumin1001> dbctl commit (dc=all): 'es1030 (re)pooling @ 75%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36405 and previous config saved to /var/cache/conftool/dbconfig/20221026-075133-root.json [production]
07:51 <marostegui@cumin1001> dbctl commit (dc=all): 'es1029 (re)pooling @ 75%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36404 and previous config saved to /var/cache/conftool/dbconfig/20221026-075126-root.json [production]
07:46 <marostegui@cumin1001> dbctl commit (dc=all): 'es2029 (re)pooling @ 25%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36403 and previous config saved to /var/cache/conftool/dbconfig/20221026-074642-root.json [production]
07:46 <marostegui@cumin1001> dbctl commit (dc=all): 'es2030 (re)pooling @ 25%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36402 and previous config saved to /var/cache/conftool/dbconfig/20221026-074634-root.json [production]
07:46 <marostegui@cumin1001> dbctl commit (dc=all): 'es2031 (re)pooling @ 25%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36401 and previous config saved to /var/cache/conftool/dbconfig/20221026-074627-root.json [production]
07:36 <marostegui@cumin1001> dbctl commit (dc=all): 'es1031 (re)pooling @ 50%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36400 and previous config saved to /var/cache/conftool/dbconfig/20221026-073633-root.json [production]
07:36 <marostegui@cumin1001> dbctl commit (dc=all): 'es1030 (re)pooling @ 50%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36399 and previous config saved to /var/cache/conftool/dbconfig/20221026-073628-root.json [production]