401-450 of 10000 results (73ms)
2024-03-05 ยง
12:15 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2218 (T343718)', diff saved to https://phabricator.wikimedia.org/P58473 and previous config saved to /var/cache/conftool/dbconfig/20240305-121546-ladsgroup.json [production]
12:07 <cgoubert@deploy2002> Started scap: (no justification provided) [production]
12:00 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2218', diff saved to https://phabricator.wikimedia.org/P58472 and previous config saved to /var/cache/conftool/dbconfig/20240305-120040-ladsgroup.json [production]
11:58 <klausman@deploy2002> helmfile [codfw] DONE helmfile.d/services/api-gateway: apply [production]
11:57 <klausman@deploy2002> helmfile [codfw] START helmfile.d/services/api-gateway: apply [production]
11:54 <klausman@deploy2002> helmfile [eqiad] DONE helmfile.d/services/api-gateway: apply [production]
11:54 <klausman@deploy2002> helmfile [eqiad] START helmfile.d/services/api-gateway: apply [production]
11:45 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2218', diff saved to https://phabricator.wikimedia.org/P58471 and previous config saved to /var/cache/conftool/dbconfig/20240305-114533-ladsgroup.json [production]
11:42 <klausman@deploy2002> helmfile [staging] DONE helmfile.d/services/api-gateway: apply [production]
11:42 <klausman@deploy2002> helmfile [staging] START helmfile.d/services/api-gateway: apply [production]
11:42 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
11:42 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
11:38 <klausman@deploy2002> helmfile [staging] DONE helmfile.d/services/api-gateway: apply [production]
11:38 <klausman@deploy2002> helmfile [staging] START helmfile.d/services/api-gateway: apply [production]
11:32 <klausman@deploy2002> helmfile [staging] DONE helmfile.d/services/api-gateway: apply [production]
11:32 <klausman@deploy2002> helmfile [staging] START helmfile.d/services/api-gateway: apply [production]
11:30 <jnuche@deploy2002> sync-world aborted: testwikis wikis to 1.42.0-wmf.21 refs T354439 (duration: 34m 15s) [production]
11:30 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2218 (T343718)', diff saved to https://phabricator.wikimedia.org/P58469 and previous config saved to /var/cache/conftool/dbconfig/20240305-113027-ladsgroup.json [production]
11:10 <ladsgroup@cumin1002> dbctl commit (dc=all): 'db2194 (re)pooling @ 100%: Maint over', diff saved to https://phabricator.wikimedia.org/P58467 and previous config saved to /var/cache/conftool/dbconfig/20240305-111031-root.json [production]
10:59 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2218 (T343718)', diff saved to https://phabricator.wikimedia.org/P58466 and previous config saved to /var/cache/conftool/dbconfig/20240305-105950-ladsgroup.json [production]
10:59 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2218.codfw.wmnet with reason: Maintenance [production]
10:59 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2218.codfw.wmnet with reason: Maintenance [production]
10:56 <jnuche@deploy2002> Started scap: testwikis wikis to 1.42.0-wmf.21 refs T354439 [production]
10:55 <ladsgroup@cumin1002> dbctl commit (dc=all): 'db2194 (re)pooling @ 75%: Maint over', diff saved to https://phabricator.wikimedia.org/P58465 and previous config saved to /var/cache/conftool/dbconfig/20240305-105526-root.json [production]
10:53 <jnuche@deploy2002> Pruned MediaWiki: 1.42.0-wmf.18 (duration: 03m 25s) [production]
10:51 <akosiaris@cumin1002> conftool action : set/pooled=yes; selector: service=kubesvc,name=parse1.* [production]
10:50 <akosiaris@cumin1002> conftool action : set/weight=10; selector: service=kubesvc,name=parse1.* [production]
10:50 <akosiaris@cumin1002> conftool action : set/pooled=yes; selector: service=kubesvc,name=parse2.* [production]
10:50 <akosiaris@cumin1002> conftool action : set/weight=10; selector: service=kubesvc,name=parse2.* [production]
10:40 <ladsgroup@cumin1002> dbctl commit (dc=all): 'db2194 (re)pooling @ 25%: Maint over', diff saved to https://phabricator.wikimedia.org/P58464 and previous config saved to /var/cache/conftool/dbconfig/20240305-104021-root.json [production]
10:25 <ladsgroup@cumin1002> dbctl commit (dc=all): 'db2194 (re)pooling @ 10%: Maint over', diff saved to https://phabricator.wikimedia.org/P58463 and previous config saved to /var/cache/conftool/dbconfig/20240305-102516-root.json [production]
10:22 <akosiaris> uncordon parse10{20..24}.eqiad.wmnet parse10{10..12}.eqiad.wmnet T358752 [production]
10:21 <akosiaris> uncordon parse20{16..20}.codfw.wmnet T358752 [production]
10:21 <akosiaris@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host parse2020.codfw.wmnet with OS bullseye [production]
10:12 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2116 (T357189)', diff saved to https://phabricator.wikimedia.org/P58462 and previous config saved to /var/cache/conftool/dbconfig/20240305-101241-arnaudb.json [production]
10:11 <akosiaris> homer commit T358752 [production]
10:07 <jnuche@deploy2002> Finished deploy [zuul/deploy@bb76c45]: test deployment for new host (duration: 00m 24s) [production]
10:06 <jnuche@deploy2002> Started deploy [zuul/deploy@bb76c45]: test deployment for new host [production]
10:04 <jnuche@deploy2002> Finished deploy [zuul/deploy@bb76c45]: test deployment for new host (duration: 00m 01s) [production]
10:04 <jnuche@deploy2002> Started deploy [zuul/deploy@bb76c45]: test deployment for new host [production]
09:42 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2116', diff saved to https://phabricator.wikimedia.org/P58460 and previous config saved to /var/cache/conftool/dbconfig/20240305-094228-arnaudb.json [production]
09:42 <akosiaris@cumin1002> START - Cookbook sre.hosts.reimage for host parse2020.codfw.wmnet with OS bullseye [production]
09:33 <akosiaris@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host parse2019.codfw.wmnet with OS bullseye [production]
09:27 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2116 (T357189)', diff saved to https://phabricator.wikimedia.org/P58459 and previous config saved to /var/cache/conftool/dbconfig/20240305-092721-arnaudb.json [production]
09:22 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db2116 (T357189)', diff saved to https://phabricator.wikimedia.org/P58458 and previous config saved to /var/cache/conftool/dbconfig/20240305-092202-arnaudb.json [production]
09:21 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2116.codfw.wmnet with reason: Maintenance [production]
09:21 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db2116.codfw.wmnet with reason: Maintenance [production]
09:21 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1235 (T357189)', diff saved to https://phabricator.wikimedia.org/P58457 and previous config saved to /var/cache/conftool/dbconfig/20240305-092140-arnaudb.json [production]
09:16 <marostegui@cumin1002> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) for hosts db2117.codfw.wmnet [production]
09:16 <marostegui@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]