301-350 of 10000 results (70ms)
2023-08-24 ยง
12:57 <fabfur@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host lvs1018.eqiad.wmnet [production]
12:56 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2130 (T344589)', diff saved to https://phabricator.wikimedia.org/P51320 and previous config saved to /var/cache/conftool/dbconfig/20230824-125607-ladsgroup.json [production]
12:56 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2130.codfw.wmnet with reason: Maintenance [production]
12:55 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2130.codfw.wmnet with reason: Maintenance [production]
12:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2116 (T344589)', diff saved to https://phabricator.wikimedia.org/P51319 and previous config saved to /var/cache/conftool/dbconfig/20230824-125542-ladsgroup.json [production]
12:54 <fabfur@cumin1001> START - Cookbook sre.hosts.reboot-single for host lvs1018.eqiad.wmnet [production]
12:49 <jiji@cumin1001> conftool action : set/pooled=false; selector: dnsdisc=kartotherian,name=eqiad [production]
12:49 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1128', diff saved to https://phabricator.wikimedia.org/P51318 and previous config saved to /var/cache/conftool/dbconfig/20230824-124942-ladsgroup.json [production]
12:48 <effie> depool kartotherian in eqiad [production]
12:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2168:3317 (T343718)', diff saved to https://phabricator.wikimedia.org/P51317 and previous config saved to /var/cache/conftool/dbconfig/20230824-124758-ladsgroup.json [production]
12:47 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2168.codfw.wmnet with reason: Maintenance [production]
12:47 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2168.codfw.wmnet with reason: Maintenance [production]
12:47 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2159 (T343718)', diff saved to https://phabricator.wikimedia.org/P51316 and previous config saved to /var/cache/conftool/dbconfig/20230824-124737-ladsgroup.json [production]
12:45 <btullis@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host an-coord1001.eqiad.wmnet [production]
12:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2116', diff saved to https://phabricator.wikimedia.org/P51315 and previous config saved to /var/cache/conftool/dbconfig/20230824-124036-ladsgroup.json [production]
12:39 <btullis@cumin1001> START - Cookbook sre.hosts.reboot-single for host an-coord1001.eqiad.wmnet [production]
12:35 <cgoubert@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
12:34 <cgoubert@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
12:34 <cgoubert@deploy1002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
12:34 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1128', diff saved to https://phabricator.wikimedia.org/P51314 and previous config saved to /var/cache/conftool/dbconfig/20230824-123436-ladsgroup.json [production]
12:34 <cgoubert@deploy1002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
12:32 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2159', diff saved to https://phabricator.wikimedia.org/P51313 and previous config saved to /var/cache/conftool/dbconfig/20230824-123231-ladsgroup.json [production]
12:25 <fabfur> errata corrige: not lvs1020 but lvs1018 [production]
12:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2116', diff saved to https://phabricator.wikimedia.org/P51312 and previous config saved to /var/cache/conftool/dbconfig/20230824-122530-ladsgroup.json [production]
12:25 <fabfur> disabling puppet and pybal on lvs1020 for reboot (T344587) [production]
12:19 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1128 (T344589)', diff saved to https://phabricator.wikimedia.org/P51311 and previous config saved to /var/cache/conftool/dbconfig/20230824-121930-ladsgroup.json [production]
12:18 <cgoubert@deploy1002> Finished scap: Redeploying mw-on-k8s - T344904 (duration: 02m 07s) [production]
12:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2159', diff saved to https://phabricator.wikimedia.org/P51310 and previous config saved to /var/cache/conftool/dbconfig/20230824-121725-ladsgroup.json [production]
12:16 <cgoubert@deploy1002> Started scap: Redeploying mw-on-k8s - T344904 [production]
12:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1128 (T344589)', diff saved to https://phabricator.wikimedia.org/P51309 and previous config saved to /var/cache/conftool/dbconfig/20230824-121158-ladsgroup.json [production]
12:11 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1128.eqiad.wmnet with reason: Maintenance [production]
12:11 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1128.eqiad.wmnet with reason: Maintenance [production]
12:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2116 (T344589)', diff saved to https://phabricator.wikimedia.org/P51308 and previous config saved to /var/cache/conftool/dbconfig/20230824-121024-ladsgroup.json [production]
12:07 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1106.eqiad.wmnet with reason: Maintenance [production]
12:06 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1106.eqiad.wmnet with reason: Maintenance [production]
12:06 <fabfur@cumin1001> END (PASS) - Cookbook sre.cdn.roll-reboot (exit_code=0) rolling reboot on A:cp-text_drmrs and A:cp [production]
12:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2175 (T344589)', diff saved to https://phabricator.wikimedia.org/P51307 and previous config saved to /var/cache/conftool/dbconfig/20230824-120611-ladsgroup.json [production]
12:03 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2116 (T344589)', diff saved to https://phabricator.wikimedia.org/P51306 and previous config saved to /var/cache/conftool/dbconfig/20230824-120352-ladsgroup.json [production]
12:03 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2116.codfw.wmnet with reason: Maintenance [production]
12:03 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2116.codfw.wmnet with reason: Maintenance [production]
12:02 <fabfur@cumin1001> END (PASS) - Cookbook sre.cdn.roll-reboot (exit_code=0) rolling reboot on A:cp-upload_drmrs and A:cp [production]
12:02 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2159 (T343718)', diff saved to https://phabricator.wikimedia.org/P51305 and previous config saved to /var/cache/conftool/dbconfig/20230824-120218-ladsgroup.json [production]
12:00 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2102.codfw.wmnet with reason: Maintenance [production]
12:00 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2102.codfw.wmnet with reason: Maintenance [production]
11:51 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance [production]
11:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2175', diff saved to https://phabricator.wikimedia.org/P51304 and previous config saved to /var/cache/conftool/dbconfig/20230824-115105-ladsgroup.json [production]
11:51 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance [production]
11:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1202 (T343718)', diff saved to https://phabricator.wikimedia.org/P51303 and previous config saved to /var/cache/conftool/dbconfig/20230824-115056-ladsgroup.json [production]
11:49 <jiji@deploy1002> helmfile [eqiad] DONE helmfile.d/services/tegola-vector-tiles: apply [production]
11:48 <jiji@deploy1002> helmfile [eqiad] START helmfile.d/services/tegola-vector-tiles: apply [production]