501-550 of 10000 results (67ms)
2022-11-29 ยง
10:19 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1194 (T323907)', diff saved to https://phabricator.wikimedia.org/P41665 and previous config saved to /var/cache/conftool/dbconfig/20221129-101958-ladsgroup.json [production]
10:17 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1118', diff saved to https://phabricator.wikimedia.org/P41664 and previous config saved to /var/cache/conftool/dbconfig/20221129-101754-marostegui.json [production]
10:15 <marostegui@cumin1001> dbctl commit (dc=all): 'db2145 (re)pooling @ 100%: After HW maintenance', diff saved to https://phabricator.wikimedia.org/P41663 and previous config saved to /var/cache/conftool/dbconfig/20221129-101554-root.json [production]
10:15 <moritzm> upgrading puppetdb2003 to bookworm T321783 [production]
10:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P41662 and previous config saved to /var/cache/conftool/dbconfig/20221129-101225-ladsgroup.json [production]
10:07 <jynus> add temporary grants to scholarships for backups on db1117, db2160 T243037 [production]
10:03 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1194 (T323907)', diff saved to https://phabricator.wikimedia.org/P41661 and previous config saved to /var/cache/conftool/dbconfig/20221129-100319-ladsgroup.json [production]
10:03 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1194.eqiad.wmnet with reason: Maintenance [production]
10:03 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db1194.eqiad.wmnet with reason: Maintenance [production]
10:03 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1191 (T323907)', diff saved to https://phabricator.wikimedia.org/P41660 and previous config saved to /var/cache/conftool/dbconfig/20221129-100258-ladsgroup.json [production]
10:02 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1118 (T321126)', diff saved to https://phabricator.wikimedia.org/P41659 and previous config saved to /var/cache/conftool/dbconfig/20221129-100248-marostegui.json [production]
10:00 <marostegui@cumin1001> dbctl commit (dc=all): 'db2145 (re)pooling @ 75%: After HW maintenance', diff saved to https://phabricator.wikimedia.org/P41658 and previous config saved to /var/cache/conftool/dbconfig/20221129-100049-root.json [production]
10:00 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1118 (T321126)', diff saved to https://phabricator.wikimedia.org/P41657 and previous config saved to /var/cache/conftool/dbconfig/20221129-100025-marostegui.json [production]
10:00 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1118.eqiad.wmnet with reason: Maintenance [production]
09:59 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1118.eqiad.wmnet with reason: Maintenance [production]
09:59 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1107 (T321126)', diff saved to https://phabricator.wikimedia.org/P41656 and previous config saved to /var/cache/conftool/dbconfig/20221129-095931-marostegui.json [production]
09:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P41655 and previous config saved to /var/cache/conftool/dbconfig/20221129-095718-ladsgroup.json [production]
09:56 <moritzm> installing curl security updates [production]
09:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2120 (T323907)', diff saved to https://phabricator.wikimedia.org/P41654 and previous config saved to /var/cache/conftool/dbconfig/20221129-094818-ladsgroup.json [production]
09:48 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2120.codfw.wmnet with reason: Maintenance [production]
09:48 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db2120.codfw.wmnet with reason: Maintenance [production]
09:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2108 (T323907)', diff saved to https://phabricator.wikimedia.org/P41653 and previous config saved to /var/cache/conftool/dbconfig/20221129-094757-ladsgroup.json [production]
09:47 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1191', diff saved to https://phabricator.wikimedia.org/P41652 and previous config saved to /var/cache/conftool/dbconfig/20221129-094745-ladsgroup.json [production]
09:45 <marostegui@cumin1001> dbctl commit (dc=all): 'db2145 (re)pooling @ 50%: After HW maintenance', diff saved to https://phabricator.wikimedia.org/P41651 and previous config saved to /var/cache/conftool/dbconfig/20221129-094544-root.json [production]
09:44 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1107', diff saved to https://phabricator.wikimedia.org/P41650 and previous config saved to /var/cache/conftool/dbconfig/20221129-094424-marostegui.json [production]
09:42 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149 (T322618)', diff saved to https://phabricator.wikimedia.org/P41649 and previous config saved to /var/cache/conftool/dbconfig/20221129-094212-ladsgroup.json [production]
09:34 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2149 (T322618)', diff saved to https://phabricator.wikimedia.org/P41648 and previous config saved to /var/cache/conftool/dbconfig/20221129-093420-ladsgroup.json [production]
09:34 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
09:34 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
09:32 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2108', diff saved to https://phabricator.wikimedia.org/P41647 and previous config saved to /var/cache/conftool/dbconfig/20221129-093250-ladsgroup.json [production]
09:32 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1191', diff saved to https://phabricator.wikimedia.org/P41646 and previous config saved to /var/cache/conftool/dbconfig/20221129-093239-ladsgroup.json [production]
09:30 <marostegui@cumin1001> dbctl commit (dc=all): 'db2145 (re)pooling @ 25%: After HW maintenance', diff saved to https://phabricator.wikimedia.org/P41645 and previous config saved to /var/cache/conftool/dbconfig/20221129-093039-root.json [production]
09:29 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1107', diff saved to https://phabricator.wikimedia.org/P41644 and previous config saved to /var/cache/conftool/dbconfig/20221129-092918-marostegui.json [production]
09:28 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2139.codfw.wmnet with reason: Maintenance [production]
09:28 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2139.codfw.wmnet with reason: Maintenance [production]
09:28 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2109 (T322618)', diff saved to https://phabricator.wikimedia.org/P41643 and previous config saved to /var/cache/conftool/dbconfig/20221129-092822-ladsgroup.json [production]
09:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2108', diff saved to https://phabricator.wikimedia.org/P41642 and previous config saved to /var/cache/conftool/dbconfig/20221129-091744-ladsgroup.json [production]
09:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1191 (T323907)', diff saved to https://phabricator.wikimedia.org/P41641 and previous config saved to /var/cache/conftool/dbconfig/20221129-091732-ladsgroup.json [production]
09:17 <moritzm> update component/puppetdb7 to puppetdb 7.11.2-3 (fixing Postgres 15 compat) T321783 [production]
09:15 <marostegui@cumin1001> dbctl commit (dc=all): 'db2145 (re)pooling @ 10%: After HW maintenance', diff saved to https://phabricator.wikimedia.org/P41640 and previous config saved to /var/cache/conftool/dbconfig/20221129-091534-root.json [production]
09:14 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1107 (T321126)', diff saved to https://phabricator.wikimedia.org/P41639 and previous config saved to /var/cache/conftool/dbconfig/20221129-091412-marostegui.json [production]
09:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2109', diff saved to https://phabricator.wikimedia.org/P41638 and previous config saved to /var/cache/conftool/dbconfig/20221129-091315-ladsgroup.json [production]
09:12 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2145', diff saved to https://phabricator.wikimedia.org/P41637 and previous config saved to /var/cache/conftool/dbconfig/20221129-091224-marostegui.json [production]
09:11 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1107 (T321126)', diff saved to https://phabricator.wikimedia.org/P41636 and previous config saved to /var/cache/conftool/dbconfig/20221129-091149-marostegui.json [production]
09:11 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1107.eqiad.wmnet with reason: Maintenance [production]
09:11 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1107.eqiad.wmnet with reason: Maintenance [production]
09:11 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106 (T321126)', diff saved to https://phabricator.wikimedia.org/P41635 and previous config saved to /var/cache/conftool/dbconfig/20221129-091117-marostegui.json [production]
09:02 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2108 (T323907)', diff saved to https://phabricator.wikimedia.org/P41634 and previous config saved to /var/cache/conftool/dbconfig/20221129-090237-ladsgroup.json [production]
09:00 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1191 (T323907)', diff saved to https://phabricator.wikimedia.org/P41633 and previous config saved to /var/cache/conftool/dbconfig/20221129-090044-ladsgroup.json [production]
09:00 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1191.eqiad.wmnet with reason: Maintenance [production]