201-250 of 10000 results (90ms)
2024-05-24 §
08:23 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1173', diff saved to https://phabricator.wikimedia.org/P63067 and previous config saved to /var/cache/conftool/dbconfig/20240524-082343-marostegui.json [production]
08:10 <logmsgbot> @deploy1002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
08:10 <logmsgbot> @deploy1002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
08:08 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T364299)', diff saved to https://phabricator.wikimedia.org/P63066 and previous config saved to /var/cache/conftool/dbconfig/20240524-080835-marostegui.json [production]
07:40 <dcausse@deploy1002> Finished deploy [airflow-dags/search@8f0b4a1]: search: fix import_ttl dag (duration: 00m 19s) [production]
07:40 <dcausse@deploy1002> Started deploy [airflow-dags/search@8f0b4a1]: search: fix import_ttl dag [production]
07:26 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1173 (T364299)', diff saved to https://phabricator.wikimedia.org/P63065 and previous config saved to /var/cache/conftool/dbconfig/20240524-072639-marostegui.json [production]
07:26 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
07:26 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
07:26 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1168 (T364299)', diff saved to https://phabricator.wikimedia.org/P63064 and previous config saved to /var/cache/conftool/dbconfig/20240524-072616-marostegui.json [production]
07:13 <logmsgbot> @deploy1002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
07:13 <logmsgbot> @deploy1002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
07:11 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1168', diff saved to https://phabricator.wikimedia.org/P63063 and previous config saved to /var/cache/conftool/dbconfig/20240524-071108-marostegui.json [production]
06:56 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1168', diff saved to https://phabricator.wikimedia.org/P63062 and previous config saved to /var/cache/conftool/dbconfig/20240524-065600-marostegui.json [production]
06:40 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1168 (T364299)', diff saved to https://phabricator.wikimedia.org/P63061 and previous config saved to /var/cache/conftool/dbconfig/20240524-064053-marostegui.json [production]
06:18 <marostegui@cumin1002> dbctl commit (dc=all): 'db2122 (re)pooling @ 100%: After reimage', diff saved to https://phabricator.wikimedia.org/P63060 and previous config saved to /var/cache/conftool/dbconfig/20240524-061812-root.json [production]
06:03 <marostegui@cumin1002> dbctl commit (dc=all): 'db2122 (re)pooling @ 75%: After reimage', diff saved to https://phabricator.wikimedia.org/P63059 and previous config saved to /var/cache/conftool/dbconfig/20240524-060305-root.json [production]
05:56 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1168 (T364299)', diff saved to https://phabricator.wikimedia.org/P63058 and previous config saved to /var/cache/conftool/dbconfig/20240524-055616-marostegui.json [production]
05:56 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1168.eqiad.wmnet with reason: Maintenance [production]
05:55 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 6:00:00 on db1168.eqiad.wmnet with reason: Maintenance [production]
05:55 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1165 (T364299)', diff saved to https://phabricator.wikimedia.org/P63057 and previous config saved to /var/cache/conftool/dbconfig/20240524-055553-marostegui.json [production]
05:48 <marostegui@cumin1002> dbctl commit (dc=all): 'db2122 (re)pooling @ 50%: After reimage', diff saved to https://phabricator.wikimedia.org/P63056 and previous config saved to /var/cache/conftool/dbconfig/20240524-054759-root.json [production]
05:40 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1165', diff saved to https://phabricator.wikimedia.org/P63055 and previous config saved to /var/cache/conftool/dbconfig/20240524-054045-marostegui.json [production]
05:32 <marostegui@cumin1002> dbctl commit (dc=all): 'db2122 (re)pooling @ 25%: After reimage', diff saved to https://phabricator.wikimedia.org/P63054 and previous config saved to /var/cache/conftool/dbconfig/20240524-053250-root.json [production]
05:25 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1165', diff saved to https://phabricator.wikimedia.org/P63053 and previous config saved to /var/cache/conftool/dbconfig/20240524-052537-marostegui.json [production]
05:23 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2122.codfw.wmnet with OS bookworm [production]
05:17 <marostegui@cumin1002> dbctl commit (dc=all): 'db2122 (re)pooling @ 10%: After reimage', diff saved to https://phabricator.wikimedia.org/P63052 and previous config saved to /var/cache/conftool/dbconfig/20240524-051744-root.json [production]
05:10 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1165 (T364299)', diff saved to https://phabricator.wikimedia.org/P63051 and previous config saved to /var/cache/conftool/dbconfig/20240524-051028-marostegui.json [production]
04:59 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2122.codfw.wmnet with reason: host reimage [production]
04:56 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on db2122.codfw.wmnet with reason: host reimage [production]
04:36 <marostegui@cumin1002> START - Cookbook sre.hosts.reimage for host db2122.codfw.wmnet with OS bookworm [production]
04:34 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2122', diff saved to https://phabricator.wikimedia.org/P63050 and previous config saved to /var/cache/conftool/dbconfig/20240524-043441-root.json [production]
04:23 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1165 (T364299)', diff saved to https://phabricator.wikimedia.org/P63049 and previous config saved to /var/cache/conftool/dbconfig/20240524-042358-marostegui.json [production]
04:23 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb[1015,1019,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance [production]
04:23 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on clouddb[1015,1019,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance [production]
04:23 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1165.eqiad.wmnet with reason: Maintenance [production]
04:23 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 6:00:00 on db1165.eqiad.wmnet with reason: Maintenance [production]
02:58 <logmsgbot> @deploy1002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
02:58 <logmsgbot> @deploy1002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
00:43 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2217 (T364299)', diff saved to https://phabricator.wikimedia.org/P63048 and previous config saved to /var/cache/conftool/dbconfig/20240524-004342-marostegui.json [production]
00:28 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2217', diff saved to https://phabricator.wikimedia.org/P63047 and previous config saved to /var/cache/conftool/dbconfig/20240524-002834-marostegui.json [production]
00:13 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2217', diff saved to https://phabricator.wikimedia.org/P63046 and previous config saved to /var/cache/conftool/dbconfig/20240524-001326-marostegui.json [production]
2024-05-23 §
23:58 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2217 (T364299)', diff saved to https://phabricator.wikimedia.org/P63045 and previous config saved to /var/cache/conftool/dbconfig/20240523-235817-marostegui.json [production]
23:30 <ladsgroup@cumin1002> dbctl commit (dc=all): 'db1238 (re)pooling @ 100%: Maint over', diff saved to https://phabricator.wikimedia.org/P63044 and previous config saved to /var/cache/conftool/dbconfig/20240523-233017-ladsgroup.json [production]
23:24 <zabe@deploy1002> Finished scap: Backport for [[gerrit:1032586|Deploy configuration for wrapping B type passwords with encrypted Argon2 (T112359)]] (duration: 16m 00s) [production]
23:15 <ladsgroup@cumin1002> dbctl commit (dc=all): 'db1238 (re)pooling @ 75%: Maint over', diff saved to https://phabricator.wikimedia.org/P63043 and previous config saved to /var/cache/conftool/dbconfig/20240523-231511-ladsgroup.json [production]
23:13 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2217 (T364299)', diff saved to https://phabricator.wikimedia.org/P63042 and previous config saved to /var/cache/conftool/dbconfig/20240523-231302-marostegui.json [production]
23:12 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2217.codfw.wmnet with reason: Maintenance [production]
23:12 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 6:00:00 on db2217.codfw.wmnet with reason: Maintenance [production]
23:12 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2214 (T364299)', diff saved to https://phabricator.wikimedia.org/P63041 and previous config saved to /var/cache/conftool/dbconfig/20240523-231238-marostegui.json [production]