3051-3100 of 10000 results (82ms)
2022-10-26 ยง
08:55 <oblivian@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
08:55 <oblivian@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
08:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2115 (T321312)', diff saved to https://phabricator.wikimedia.org/P36424 and previous config saved to /var/cache/conftool/dbconfig/20221026-085257-ladsgroup.json [production]
08:52 <jmm@cumin2002> START - Cookbook sre.hosts.reimage for host ganeti1015.eqiad.wmnet with OS bullseye [production]
08:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1129 (T321312)', diff saved to https://phabricator.wikimedia.org/P36423 and previous config saved to /var/cache/conftool/dbconfig/20221026-085022-ladsgroup.json [production]
08:50 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
08:50 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
08:49 <ladsgroup@cumin1001> dbctl commit (dc=all): 'db1160 (re)pooling @ 10%: Maint finished', diff saved to https://phabricator.wikimedia.org/P36422 and previous config saved to /var/cache/conftool/dbconfig/20221026-084954-ladsgroup.json [production]
08:49 <ladsgroup@cumin1001> dbctl commit (dc=all): 'db1160 (re)pooling @ 25%: 10', diff saved to https://phabricator.wikimedia.org/P36421 and previous config saved to /var/cache/conftool/dbconfig/20221026-084922-ladsgroup.json [production]
08:47 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2115 (T321312)', diff saved to https://phabricator.wikimedia.org/P36420 and previous config saved to /var/cache/conftool/dbconfig/20221026-084741-ladsgroup.json [production]
08:47 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2115.codfw.wmnet with reason: Maintenance [production]
08:47 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2115.codfw.wmnet with reason: Maintenance [production]
08:46 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
08:46 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
08:31 <marostegui@cumin1001> dbctl commit (dc=all): 'es2029 (re)pooling @ 100%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36418 and previous config saved to /var/cache/conftool/dbconfig/20221026-083157-root.json [production]
08:31 <marostegui@cumin1001> dbctl commit (dc=all): 'es2030 (re)pooling @ 100%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36417 and previous config saved to /var/cache/conftool/dbconfig/20221026-083149-root.json [production]
08:31 <marostegui@cumin1001> dbctl commit (dc=all): 'es2031 (re)pooling @ 100%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36416 and previous config saved to /var/cache/conftool/dbconfig/20221026-083142-root.json [production]
08:25 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on ganeti1015.eqiad.wmnet with reason: Remove from cluster for eventual reimage [production]
08:25 <jmm@cumin2002> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on ganeti1015.eqiad.wmnet with reason: Remove from cluster for eventual reimage [production]
08:20 <oblivian@deploy1002> helmfile [codfw] DONE helmfile.d/services/shellbox-timeline: apply [production]
08:19 <oblivian@deploy1002> helmfile [codfw] START helmfile.d/services/shellbox-timeline: apply [production]
08:18 <oblivian@deploy1002> helmfile [eqiad] DONE helmfile.d/services/shellbox-timeline: apply [production]
08:18 <oblivian@deploy1002> helmfile [eqiad] START helmfile.d/services/shellbox-timeline: apply [production]
08:18 <oblivian@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-timeline: apply [production]
08:17 <oblivian@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-timeline: apply [production]
08:16 <marostegui@cumin1001> dbctl commit (dc=all): 'es2029 (re)pooling @ 75%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36415 and previous config saved to /var/cache/conftool/dbconfig/20221026-081652-root.json [production]
08:16 <marostegui@cumin1001> dbctl commit (dc=all): 'es2030 (re)pooling @ 75%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36414 and previous config saved to /var/cache/conftool/dbconfig/20221026-081644-root.json [production]
08:16 <marostegui@cumin1001> dbctl commit (dc=all): 'es2031 (re)pooling @ 75%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36413 and previous config saved to /var/cache/conftool/dbconfig/20221026-081637-root.json [production]
08:16 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
08:15 <oblivian@deploy1002> helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply [production]
08:14 <oblivian@deploy1002> helmfile [eqiad] START helmfile.d/services/shellbox-media: apply [production]
08:12 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
08:12 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
08:12 <jnuche@deploy1002> Synchronized php: group1 wikis to 1.40.0-wmf.7 refs T320512 (duration: 03m 46s) [production]
08:12 <oblivian@deploy1002> helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply [production]
08:11 <oblivian@deploy1002> helmfile [codfw] START helmfile.d/services/shellbox-media: apply [production]
08:11 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
08:11 <oblivian@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-media: apply [production]
08:10 <oblivian@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-media: apply [production]
08:08 <jnuche@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.40.0-wmf.7 refs T320512 [production]
08:06 <marostegui@cumin1001> dbctl commit (dc=all): 'es1031 (re)pooling @ 100%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36412 and previous config saved to /var/cache/conftool/dbconfig/20221026-080643-root.json [production]
08:06 <marostegui@cumin1001> dbctl commit (dc=all): 'es1030 (re)pooling @ 100%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36411 and previous config saved to /var/cache/conftool/dbconfig/20221026-080638-root.json [production]
08:06 <marostegui@cumin1001> dbctl commit (dc=all): 'es1029 (re)pooling @ 100%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36410 and previous config saved to /var/cache/conftool/dbconfig/20221026-080631-root.json [production]
08:02 <oblivian@deploy1002> helmfile [eqiad] DONE helmfile.d/services/shellbox-constraints: apply [production]
08:02 <oblivian@deploy1002> helmfile [eqiad] START helmfile.d/services/shellbox-constraints: apply [production]
08:01 <marostegui@cumin1001> dbctl commit (dc=all): 'es2029 (re)pooling @ 50%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36409 and previous config saved to /var/cache/conftool/dbconfig/20221026-080147-root.json [production]
08:01 <marostegui@cumin1001> dbctl commit (dc=all): 'es2030 (re)pooling @ 50%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36408 and previous config saved to /var/cache/conftool/dbconfig/20221026-080139-root.json [production]
08:01 <marostegui@cumin1001> dbctl commit (dc=all): 'es2031 (re)pooling @ 50%: After upgrade', diff saved to https://phabricator.wikimedia.org/P36407 and previous config saved to /var/cache/conftool/dbconfig/20221026-080132-root.json [production]
08:00 <oblivian@deploy1002> helmfile [codfw] DONE helmfile.d/services/shellbox-constraints: apply [production]
08:00 <oblivian@deploy1002> helmfile [codfw] START helmfile.d/services/shellbox-constraints: apply [production]