4101-4150 of 10000 results (79ms)
2022-11-30 ยง
12:47 <kartik@deploy1002> helmfile [staging] START helmfile.d/services/cxserver: apply [production]
12:45 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2097.codfw.wmnet with reason: Maintenance [production]
12:44 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db2097.codfw.wmnet with reason: Maintenance [production]
12:34 <btullis@cumin1001> END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) [production]
12:29 <btullis@cumin1001> START - Cookbook sre.discovery.service-route [production]
12:27 <btullis@deploy1002> helmfile [eqiad] DONE helmfile.d/services/eventstreams: apply [production]
12:26 <btullis@deploy1002> helmfile [eqiad] START helmfile.d/services/eventstreams: apply [production]
12:24 <btullis@deploy1002> helmfile [eqiad] DONE helmfile.d/services/eventstreams: apply [production]
12:24 <btullis@deploy1002> helmfile [eqiad] START helmfile.d/services/eventstreams: apply [production]
12:13 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
12:12 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
12:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197 (T323907)', diff saved to https://phabricator.wikimedia.org/P41879 and previous config saved to /var/cache/conftool/dbconfig/20221130-121246-ladsgroup.json [production]
12:03 <btullis@cumin1001> END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) [production]
12:00 <oblivian@deploy1002> helmfile [eqiad] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
11:59 <oblivian@deploy1002> helmfile [eqiad] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
11:58 <btullis@cumin1001> START - Cookbook sre.discovery.service-route [production]
11:58 <oblivian@deploy1002> helmfile [codfw] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
11:57 <oblivian@deploy1002> helmfile [codfw] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
11:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197', diff saved to https://phabricator.wikimedia.org/P41877 and previous config saved to /var/cache/conftool/dbconfig/20221130-115739-ladsgroup.json [production]
11:56 <oblivian@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
11:56 <oblivian@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
11:53 <oblivian@deploy1002> helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply [production]
11:53 <btullis@cumin1001> END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) [production]
11:53 <btullis@cumin1001> START - Cookbook sre.discovery.service-route [production]
11:50 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host puppetdb1003.eqiad.wmnet [production]
11:50 <oblivian@deploy1002> helmfile [eqiad] START helmfile.d/services/shellbox-media: apply [production]
11:50 <oblivian@deploy1002> helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply [production]
11:48 <oblivian@deploy1002> helmfile [codfw] START helmfile.d/services/shellbox-media: apply [production]
11:44 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host puppetdb1003.eqiad.wmnet [production]
11:42 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197', diff saved to https://phabricator.wikimedia.org/P41876 and previous config saved to /var/cache/conftool/dbconfig/20221130-114233-ladsgroup.json [production]
11:40 <oblivian@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-media: apply [production]
11:40 <oblivian@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-media: apply [production]
11:35 <btullis@puppetmaster1001> conftool action : set/pooled=true; selector: dnsdisc=eventstreams,name=codfw [production]
11:32 <btullis@deploy1002> helmfile [codfw] DONE helmfile.d/services/eventstreams: apply [production]
11:32 <btullis@deploy1002> helmfile [codfw] START helmfile.d/services/eventstreams: apply [production]
11:27 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197 (T323907)', diff saved to https://phabricator.wikimedia.org/P41875 and previous config saved to /var/cache/conftool/dbconfig/20221130-112726-ladsgroup.json [production]
11:19 <moritzm> upgrade puppetdb1003 to bookworm T321783 [production]
11:14 <oblivian@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-constraints: apply [production]
11:14 <oblivian@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-constraints: apply [production]
11:13 <oblivian@deploy1002> helmfile [eqiad] DONE helmfile.d/services/shellbox-constraints: apply [production]
11:12 <oblivian@deploy1002> helmfile [eqiad] START helmfile.d/services/shellbox-constraints: apply [production]
11:12 <oblivian@deploy1002> helmfile [codfw] DONE helmfile.d/services/shellbox-constraints: apply [production]
11:12 <btullis@puppetmaster1001> conftool action : set/pooled=false; selector: dnsdisc=eventstreams,name=codfw [production]
11:11 <oblivian@deploy1002> helmfile [codfw] START helmfile.d/services/shellbox-constraints: apply [production]
11:11 <oblivian@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-constraints: apply [production]
11:11 <oblivian@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-constraints: apply [production]
11:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1197 (T323907)', diff saved to https://phabricator.wikimedia.org/P41874 and previous config saved to /var/cache/conftool/dbconfig/20221130-111003-ladsgroup.json [production]
11:09 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1197.eqiad.wmnet with reason: Maintenance [production]
11:09 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db1197.eqiad.wmnet with reason: Maintenance [production]
11:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1188 (T323907)', diff saved to https://phabricator.wikimedia.org/P41873 and previous config saved to /var/cache/conftool/dbconfig/20221130-110942-ladsgroup.json [production]