2051-2100 of 10000 results (77ms)
2022-10-01 §
13:24 <fab@deploy1002> Started deploy [airflow-dags/research@44a1158]: (no justification provided) [production]
13:12 <fab@deploy1002> Finished deploy [airflow-dags/research@d6b3e82]: (no justification provided) (duration: 03m 35s) [production]
13:08 <fab@deploy1002> Started deploy [airflow-dags/research@d6b3e82]: (no justification provided) [production]
05:17 <JJMC89> disable all plagiabot jobs T319123 [tools.eranbot]
01:16 <wm-bot> <anticomposite> ./stewardbots/SULWatcher/manage.sh restart [tools.stewardbots]
2022-09-30 §
23:26 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance [production]
23:25 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance [production]
23:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1196 (T314041)', diff saved to https://phabricator.wikimedia.org/P35243 and previous config saved to /var/cache/conftool/dbconfig/20220930-232546-ladsgroup.json [production]
23:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1196', diff saved to https://phabricator.wikimedia.org/P35242 and previous config saved to /var/cache/conftool/dbconfig/20220930-231040-ladsgroup.json [production]
22:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1196', diff saved to https://phabricator.wikimedia.org/P35241 and previous config saved to /var/cache/conftool/dbconfig/20220930-225534-ladsgroup.json [production]
22:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1196 (T314041)', diff saved to https://phabricator.wikimedia.org/P35240 and previous config saved to /var/cache/conftool/dbconfig/20220930-224027-ladsgroup.json [production]
21:02 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudbackup2001.codfw.wmnet [production]
20:54 <andrew@cumin1001> START - Cookbook sre.hosts.reboot-single for host cloudbackup2001.codfw.wmnet [production]
18:43 <James_F> Triggering graceful restart of zuul to see if that fixes on-going merge/gerrit connection issues. [releng]
18:30 <robh@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp4045.ulsfo.wmnet with OS bullseye [production]
18:08 <robh@cumin2002> START - Cookbook sre.hosts.reimage for host cp4045.ulsfo.wmnet with OS bullseye [production]
18:01 <robh@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp4045.ulsfo.wmnet with OS bullseye [production]
17:43 <robh@cumin2002> START - Cookbook sre.hosts.reimage for host cp4045.ulsfo.wmnet with OS bullseye [production]
17:24 <bblack@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=93) for host cp4045.ulsfo.wmnet with OS bullseye [production]
17:07 <James_F> Zuul: Make PHP 8.1 non-voting for all skins and extensions T316078 [releng]
17:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1196 (T314041)', diff saved to https://phabricator.wikimedia.org/P35237 and previous config saved to /var/cache/conftool/dbconfig/20220930-170620-ladsgroup.json [production]
17:06 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1196.eqiad.wmnet with reason: Maintenance [production]
17:05 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1196.eqiad.wmnet with reason: Maintenance [production]
17:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1186 (T314041)', diff saved to https://phabricator.wikimedia.org/P35236 and previous config saved to /var/cache/conftool/dbconfig/20220930-170546-ladsgroup.json [production]
16:54 <bblack@cumin2002> START - Cookbook sre.hosts.reimage for host cp4045.ulsfo.wmnet with OS bullseye [production]
16:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1186', diff saved to https://phabricator.wikimedia.org/P35235 and previous config saved to /var/cache/conftool/dbconfig/20220930-165040-ladsgroup.json [production]
16:46 <James_F> Zuul: Make PHP 8.0 and PHP 8.1 voting for all skins and extensions in master for T300463 and T316078 [releng]
16:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1186', diff saved to https://phabricator.wikimedia.org/P35234 and previous config saved to /var/cache/conftool/dbconfig/20220930-163533-ladsgroup.json [production]
16:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1186 (T314041)', diff saved to https://phabricator.wikimedia.org/P35233 and previous config saved to /var/cache/conftool/dbconfig/20220930-162027-ladsgroup.json [production]
15:37 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1023.eqiad.wmnet with OS bullseye [production]
15:12 <James_F> Docker: Building and publishing PHP 8.0.24 images for T315167 [releng]
14:52 <wm-bot2> Added 1 new OSDs ['cloudcephosd1031.eqiad.wmnet'] - cookbook ran by fran@wmf3169 [admin]
14:52 <wm-bot2> Added OSD cloudcephosd1031.eqiad.wmnet... (1/1) - cookbook ran by fran@wmf3169 [admin]
14:48 <wm-bot2> Adding OSD cloudcephosd1031.eqiad.wmnet... (1/1) - cookbook ran by fran@wmf3169 [admin]
14:48 <wm-bot2> Adding new OSDs ['cloudcephosd1031.eqiad.wmnet'] to the cluster - cookbook ran by fran@wmf3169 [admin]
14:41 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1023.eqiad.wmnet with OS bullseye [production]
14:16 <wm-bot2> Drained 'cloudvirt1023.eqiad.wmnet'. (T319025) - cookbook ran by andrew@buster [admin]
14:15 <wm-bot2> Set cloudvirt 'cloudvirt1023.eqiad.wmnet' maintenance (downtime id: 64eac5c6-4b1d-4269-98fd-8e5bed42ce40, use this to unset). (T319025) - cookbook ran by andrew@buster [admin]
14:15 <wm-bot2> Draining 'cloudvirt1023.eqiad.wmnet'. (T319025) - cookbook ran by andrew@buster [admin]
14:15 <wm-bot2> Safe rebooting 'cloudvirt1023.eqiad.wmnet'. (T319025) - cookbook ran by andrew@buster [admin]
14:09 <wm-bot2> Drained 'cloudvirt1023.eqiad.wmnet'. - cookbook ran by andrew@buster [admin]
14:05 <wm-bot2> Set cloudvirt 'cloudvirt1023.eqiad.wmnet' maintenance (downtime id: fb99c967-b974-4314-a2fa-31ed0e883dd3, use this to unset). - cookbook ran by andrew@buster [admin]
14:04 <wm-bot2> Draining 'cloudvirt1023.eqiad.wmnet'. - cookbook ran by andrew@buster [admin]
13:51 <moritzm> installing puppetdb-test2001 T318931 [production]
13:30 <wm-bot2> Added 1 new OSDs ['cloudcephosd1031.eqiad.wmnet'] - cookbook ran by fran@wmf3169 [admin]
13:30 <wm-bot2> Added OSD cloudcephosd1031.eqiad.wmnet... (1/1) - cookbook ran by fran@wmf3169 [admin]
13:26 <wm-bot2> Adding OSD cloudcephosd1031.eqiad.wmnet... (1/1) - cookbook ran by fran@wmf3169 [admin]
13:26 <wm-bot2> Adding new OSDs ['cloudcephosd1031.eqiad.wmnet'] to the cluster - cookbook ran by fran@wmf3169 [admin]
13:23 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
13:23 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]