1151-1200 of 10000 results (66ms)
2022-10-03 §
06:51 <marostegui@cumin1001> dbctl commit (dc=all): 'db1182 (re)pooling @ 1%: After upgrade to 10.6', diff saved to https://phabricator.wikimedia.org/P35261 and previous config saved to /var/cache/conftool/dbconfig/20221003-065154-root.json [production]
06:51 <marostegui@cumin1001> dbctl commit (dc=all): 'db1167 (re)pooling @ 25%: After upgrade', diff saved to https://phabricator.wikimedia.org/P35260 and previous config saved to /var/cache/conftool/dbconfig/20221003-065112-root.json [production]
06:50 <marostegui@cumin1001> dbctl commit (dc=all): 'db1158 (re)pooling @ 5%: After upgrade', diff saved to https://phabricator.wikimedia.org/P35259 and previous config saved to /var/cache/conftool/dbconfig/20221003-065031-root.json [production]
06:48 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'configure' for AS: 6128 [production]
06:46 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1182', diff saved to https://phabricator.wikimedia.org/P35258 and previous config saved to /var/cache/conftool/dbconfig/20221003-064638-root.json [production]
06:42 <marostegui@cumin1001> dbctl commit (dc=all): 'db1179 (re)pooling @ 25%: After upgrade', diff saved to https://phabricator.wikimedia.org/P35257 and previous config saved to /var/cache/conftool/dbconfig/20221003-064226-root.json [production]
06:36 <marostegui@cumin1001> dbctl commit (dc=all): 'db1167 (re)pooling @ 10%: After upgrade', diff saved to https://phabricator.wikimedia.org/P35256 and previous config saved to /var/cache/conftool/dbconfig/20221003-063607-root.json [production]
06:35 <marostegui@cumin1001> dbctl commit (dc=all): 'db1158 (re)pooling @ 3%: After upgrade', diff saved to https://phabricator.wikimedia.org/P35255 and previous config saved to /var/cache/conftool/dbconfig/20221003-063527-root.json [production]
06:30 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 11039 [production]
06:30 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'configure' for AS: 11039 [production]
06:27 <marostegui@cumin1001> dbctl commit (dc=all): 'db1179 (re)pooling @ 10%: After upgrade', diff saved to https://phabricator.wikimedia.org/P35254 and previous config saved to /var/cache/conftool/dbconfig/20221003-062721-root.json [production]
06:27 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 5400 [production]
06:26 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'configure' for AS: 5400 [production]
06:21 <marostegui@cumin1001> dbctl commit (dc=all): 'db1167 (re)pooling @ 5%: After upgrade', diff saved to https://phabricator.wikimedia.org/P35253 and previous config saved to /var/cache/conftool/dbconfig/20221003-062102-root.json [production]
06:20 <marostegui@cumin1001> dbctl commit (dc=all): 'db1158 (re)pooling @ 1%: After upgrade', diff saved to https://phabricator.wikimedia.org/P35252 and previous config saved to /var/cache/conftool/dbconfig/20221003-062022-root.json [production]
06:15 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 3300 [production]
06:13 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'configure' for AS: 3300 [production]
06:12 <marostegui@cumin1001> dbctl commit (dc=all): 'db1179 (re)pooling @ 5%: After upgrade', diff saved to https://phabricator.wikimedia.org/P35251 and previous config saved to /var/cache/conftool/dbconfig/20221003-061216-root.json [production]
06:07 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 15133 [production]
06:05 <marostegui@cumin1001> dbctl commit (dc=all): 'db1167 (re)pooling @ 3%: After upgrade', diff saved to https://phabricator.wikimedia.org/P35250 and previous config saved to /var/cache/conftool/dbconfig/20221003-060557-root.json [production]
06:04 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'configure' for AS: 15133 [production]
05:57 <marostegui@cumin1001> dbctl commit (dc=all): 'db1179 (re)pooling @ 3%: After upgrade', diff saved to https://phabricator.wikimedia.org/P35249 and previous config saved to /var/cache/conftool/dbconfig/20221003-055711-root.json [production]
05:54 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1158', diff saved to https://phabricator.wikimedia.org/P35248 and previous config saved to /var/cache/conftool/dbconfig/20221003-055401-root.json [production]
05:50 <marostegui@cumin1001> dbctl commit (dc=all): 'db1167 (re)pooling @ 1%: After upgrade', diff saved to https://phabricator.wikimedia.org/P35247 and previous config saved to /var/cache/conftool/dbconfig/20221003-055052-root.json [production]
05:42 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1167', diff saved to https://phabricator.wikimedia.org/P35246 and previous config saved to /var/cache/conftool/dbconfig/20221003-054245-root.json [production]
05:42 <marostegui@cumin1001> dbctl commit (dc=all): 'db1179 (re)pooling @ 1%: After upgrade', diff saved to https://phabricator.wikimedia.org/P35245 and previous config saved to /var/cache/conftool/dbconfig/20221003-054206-root.json [production]
05:29 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1179', diff saved to https://phabricator.wikimedia.org/P35244 and previous config saved to /var/cache/conftool/dbconfig/20221003-052927-root.json [production]
2022-10-02 §
08:13 <elukey> `apt-get clean` on an-airflow1001 to free some space on the root partition [production]
2022-10-01 §
13:24 <fab@deploy1002> Finished deploy [airflow-dags/research@44a1158]: (no justification provided) (duration: 00m 08s) [production]
13:24 <fab@deploy1002> Started deploy [airflow-dags/research@44a1158]: (no justification provided) [production]
13:12 <fab@deploy1002> Finished deploy [airflow-dags/research@d6b3e82]: (no justification provided) (duration: 03m 35s) [production]
13:08 <fab@deploy1002> Started deploy [airflow-dags/research@d6b3e82]: (no justification provided) [production]
2022-09-30 §
23:26 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance [production]
23:25 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance [production]
23:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1196 (T314041)', diff saved to https://phabricator.wikimedia.org/P35243 and previous config saved to /var/cache/conftool/dbconfig/20220930-232546-ladsgroup.json [production]
23:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1196', diff saved to https://phabricator.wikimedia.org/P35242 and previous config saved to /var/cache/conftool/dbconfig/20220930-231040-ladsgroup.json [production]
22:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1196', diff saved to https://phabricator.wikimedia.org/P35241 and previous config saved to /var/cache/conftool/dbconfig/20220930-225534-ladsgroup.json [production]
22:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1196 (T314041)', diff saved to https://phabricator.wikimedia.org/P35240 and previous config saved to /var/cache/conftool/dbconfig/20220930-224027-ladsgroup.json [production]
21:02 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudbackup2001.codfw.wmnet [production]
20:54 <andrew@cumin1001> START - Cookbook sre.hosts.reboot-single for host cloudbackup2001.codfw.wmnet [production]
18:30 <robh@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp4045.ulsfo.wmnet with OS bullseye [production]
18:08 <robh@cumin2002> START - Cookbook sre.hosts.reimage for host cp4045.ulsfo.wmnet with OS bullseye [production]
18:01 <robh@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp4045.ulsfo.wmnet with OS bullseye [production]
17:43 <robh@cumin2002> START - Cookbook sre.hosts.reimage for host cp4045.ulsfo.wmnet with OS bullseye [production]
17:24 <bblack@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=93) for host cp4045.ulsfo.wmnet with OS bullseye [production]
17:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1196 (T314041)', diff saved to https://phabricator.wikimedia.org/P35237 and previous config saved to /var/cache/conftool/dbconfig/20220930-170620-ladsgroup.json [production]
17:06 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1196.eqiad.wmnet with reason: Maintenance [production]
17:05 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1196.eqiad.wmnet with reason: Maintenance [production]
17:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1186 (T314041)', diff saved to https://phabricator.wikimedia.org/P35236 and previous config saved to /var/cache/conftool/dbconfig/20220930-170546-ladsgroup.json [production]
16:54 <bblack@cumin2002> START - Cookbook sre.hosts.reimage for host cp4045.ulsfo.wmnet with OS bullseye [production]