1001-1050 of 10000 results (72ms)
2022-11-14 ยง
11:24 <ladsgroup@deploy1002> ladsgroup and ladsgroup: Backport for [[gerrit:854509|Re-add s11 in db config reload callback (T322598)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
11:24 <ladsgroup@deploy1002> Started scap: Backport for [[gerrit:854509|Re-add s11 in db config reload callback (T322598)]] [production]
11:23 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2125', diff saved to https://phabricator.wikimedia.org/P39430 and previous config saved to /var/cache/conftool/dbconfig/20221114-112330-ladsgroup.json [production]
11:14 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1144:3314 (T321130)', diff saved to https://phabricator.wikimedia.org/P39429 and previous config saved to /var/cache/conftool/dbconfig/20221114-111434-marostegui.json [production]
11:14 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1144.eqiad.wmnet with reason: Maintenance [production]
11:14 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1144.eqiad.wmnet with reason: Maintenance [production]
11:14 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1143 (T321130)', diff saved to https://phabricator.wikimedia.org/P39428 and previous config saved to /var/cache/conftool/dbconfig/20221114-111412-marostegui.json [production]
11:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P39427 and previous config saved to /var/cache/conftool/dbconfig/20221114-111229-ladsgroup.json [production]
11:12 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3316', diff saved to https://phabricator.wikimedia.org/P39426 and previous config saved to /var/cache/conftool/dbconfig/20221114-111222-marostegui.json [production]
11:08 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2125', diff saved to https://phabricator.wikimedia.org/P39425 and previous config saved to /var/cache/conftool/dbconfig/20221114-110824-ladsgroup.json [production]
10:59 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1143', diff saved to https://phabricator.wikimedia.org/P39424 and previous config saved to /var/cache/conftool/dbconfig/20221114-105906-marostegui.json [production]
10:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129 (T322618)', diff saved to https://phabricator.wikimedia.org/P39423 and previous config saved to /var/cache/conftool/dbconfig/20221114-105723-ladsgroup.json [production]
10:57 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3316', diff saved to https://phabricator.wikimedia.org/P39422 and previous config saved to /var/cache/conftool/dbconfig/20221114-105716-marostegui.json [production]
10:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1129 (T322618)', diff saved to https://phabricator.wikimedia.org/P39421 and previous config saved to /var/cache/conftool/dbconfig/20221114-105512-ladsgroup.json [production]
10:55 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
10:54 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
10:54 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3312 (T322618)', diff saved to https://phabricator.wikimedia.org/P39420 and previous config saved to /var/cache/conftool/dbconfig/20221114-105450-ladsgroup.json [production]
10:53 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2125 (T322618)', diff saved to https://phabricator.wikimedia.org/P39419 and previous config saved to /var/cache/conftool/dbconfig/20221114-105317-ladsgroup.json [production]
10:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2125 (T322618)', diff saved to https://phabricator.wikimedia.org/P39418 and previous config saved to /var/cache/conftool/dbconfig/20221114-105056-ladsgroup.json [production]
10:50 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2125.codfw.wmnet with reason: Maintenance [production]
10:50 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2125.codfw.wmnet with reason: Maintenance [production]
10:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2104 (T322618)', diff saved to https://phabricator.wikimedia.org/P39417 and previous config saved to /var/cache/conftool/dbconfig/20221114-105034-ladsgroup.json [production]
10:44 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1143', diff saved to https://phabricator.wikimedia.org/P39416 and previous config saved to /var/cache/conftool/dbconfig/20221114-104400-marostegui.json [production]
10:42 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3316 (T321126)', diff saved to https://phabricator.wikimedia.org/P39415 and previous config saved to /var/cache/conftool/dbconfig/20221114-104209-marostegui.json [production]
10:39 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1096:3316 (T321126)', diff saved to https://phabricator.wikimedia.org/P39414 and previous config saved to /var/cache/conftool/dbconfig/20221114-103953-marostegui.json [production]
10:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3312', diff saved to https://phabricator.wikimedia.org/P39413 and previous config saved to /var/cache/conftool/dbconfig/20221114-103944-ladsgroup.json [production]
10:39 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1096.eqiad.wmnet with reason: Maintenance [production]
10:39 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1096.eqiad.wmnet with reason: Maintenance [production]
10:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2104', diff saved to https://phabricator.wikimedia.org/P39412 and previous config saved to /var/cache/conftool/dbconfig/20221114-103528-ladsgroup.json [production]
10:28 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1143 (T321130)', diff saved to https://phabricator.wikimedia.org/P39411 and previous config saved to /var/cache/conftool/dbconfig/20221114-102853-marostegui.json [production]
10:24 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3312', diff saved to https://phabricator.wikimedia.org/P39410 and previous config saved to /var/cache/conftool/dbconfig/20221114-102437-ladsgroup.json [production]
10:21 <marostegui@cumin1001> dbctl commit (dc=all): 'db2145 (re)pooling @ 100%: After upgrade', diff saved to https://phabricator.wikimedia.org/P39409 and previous config saved to /var/cache/conftool/dbconfig/20221114-102155-root.json [production]
10:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2104', diff saved to https://phabricator.wikimedia.org/P39408 and previous config saved to /var/cache/conftool/dbconfig/20221114-102021-ladsgroup.json [production]
10:16 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1143 (T321130)', diff saved to https://phabricator.wikimedia.org/P39407 and previous config saved to /var/cache/conftool/dbconfig/20221114-101659-marostegui.json [production]
10:16 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1143.eqiad.wmnet with reason: Maintenance [production]
10:16 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1143.eqiad.wmnet with reason: Maintenance [production]
10:16 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1142 (T321130)', diff saved to https://phabricator.wikimedia.org/P39406 and previous config saved to /var/cache/conftool/dbconfig/20221114-101637-marostegui.json [production]
10:12 <vgutierrez> upgrading acme-chief on acmechief1001 to version 0.35 (requires disabling puppet on R:acme_chief::cert) [production]
10:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3312 (T322618)', diff saved to https://phabricator.wikimedia.org/P39405 and previous config saved to /var/cache/conftool/dbconfig/20221114-100931-ladsgroup.json [production]
10:09 <ladsgroup@deploy1002> Finished scap: Backport for [[gerrit:855743|Rework SpecialPagesWithoutScans query (T322849)]] (duration: 11m 17s) [production]
10:07 <vgutierrez> upload acme-chief 0.35 to apt.wm.o (buster-wikimedia) - T244232 T262251 [production]
10:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1105:3312 (T322618)', diff saved to https://phabricator.wikimedia.org/P39404 and previous config saved to /var/cache/conftool/dbconfig/20221114-100720-ladsgroup.json [production]
10:07 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1105.eqiad.wmnet with reason: Maintenance [production]
10:07 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1105.eqiad.wmnet with reason: Maintenance [production]
10:06 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
10:06 <marostegui@cumin1001> dbctl commit (dc=all): 'db2145 (re)pooling @ 75%: After upgrade', diff saved to https://phabricator.wikimedia.org/P39403 and previous config saved to /var/cache/conftool/dbconfig/20221114-100650-root.json [production]
10:06 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
10:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2104 (T322618)', diff saved to https://phabricator.wikimedia.org/P39402 and previous config saved to /var/cache/conftool/dbconfig/20221114-100515-ladsgroup.json [production]
10:02 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2104 (T322618)', diff saved to https://phabricator.wikimedia.org/P39401 and previous config saved to /var/cache/conftool/dbconfig/20221114-100254-ladsgroup.json [production]
10:02 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2104.codfw.wmnet with reason: Maintenance [production]