1901-1950 of 10000 results (94ms)
2025-08-02 ยง
17:49 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2190.codfw.wmnet with reason: Maintenance [production]
17:49 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2177 (T400854)', diff saved to https://phabricator.wikimedia.org/P80525 and previous config saved to /var/cache/conftool/dbconfig/20250802-174929-ladsgroup.json [production]
17:34 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2177', diff saved to https://phabricator.wikimedia.org/P80524 and previous config saved to /var/cache/conftool/dbconfig/20250802-173422-ladsgroup.json [production]
17:33 <hnowlan> clean up some misbehaving thumbor pods [production]
17:19 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2177', diff saved to https://phabricator.wikimedia.org/P80523 and previous config saved to /var/cache/conftool/dbconfig/20250802-171914-ladsgroup.json [production]
17:04 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2177 (T400854)', diff saved to https://phabricator.wikimedia.org/P80522 and previous config saved to /var/cache/conftool/dbconfig/20250802-170407-ladsgroup.json [production]
16:50 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2177 (T400854)', diff saved to https://phabricator.wikimedia.org/P80521 and previous config saved to /var/cache/conftool/dbconfig/20250802-165012-ladsgroup.json [production]
16:50 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2177.codfw.wmnet with reason: Maintenance [production]
16:49 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2156 (T400854)', diff saved to https://phabricator.wikimedia.org/P80520 and previous config saved to /var/cache/conftool/dbconfig/20250802-164949-ladsgroup.json [production]
16:34 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P80519 and previous config saved to /var/cache/conftool/dbconfig/20250802-163441-ladsgroup.json [production]
16:19 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P80518 and previous config saved to /var/cache/conftool/dbconfig/20250802-161933-ladsgroup.json [production]
16:04 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2156 (T400854)', diff saved to https://phabricator.wikimedia.org/P80517 and previous config saved to /var/cache/conftool/dbconfig/20250802-160426-ladsgroup.json [production]
15:50 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2156 (T400854)', diff saved to https://phabricator.wikimedia.org/P80516 and previous config saved to /var/cache/conftool/dbconfig/20250802-155032-ladsgroup.json [production]
15:50 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
15:50 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2149 (T400854)', diff saved to https://phabricator.wikimedia.org/P80515 and previous config saved to /var/cache/conftool/dbconfig/20250802-155008-ladsgroup.json [production]
15:35 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P80514 and previous config saved to /var/cache/conftool/dbconfig/20250802-153501-ladsgroup.json [production]
15:19 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P80513 and previous config saved to /var/cache/conftool/dbconfig/20250802-151953-ladsgroup.json [production]
15:04 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2149 (T400854)', diff saved to https://phabricator.wikimedia.org/P80512 and previous config saved to /var/cache/conftool/dbconfig/20250802-150446-ladsgroup.json [production]
14:50 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2149 (T400854)', diff saved to https://phabricator.wikimedia.org/P80511 and previous config saved to /var/cache/conftool/dbconfig/20250802-145049-ladsgroup.json [production]
14:50 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
14:47 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
14:43 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1240.eqiad.wmnet with reason: Maintenance [production]
14:43 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1212 (T400854)', diff saved to https://phabricator.wikimedia.org/P80510 and previous config saved to /var/cache/conftool/dbconfig/20250802-144311-ladsgroup.json [production]
14:28 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1212', diff saved to https://phabricator.wikimedia.org/P80509 and previous config saved to /var/cache/conftool/dbconfig/20250802-142803-ladsgroup.json [production]
14:12 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1212', diff saved to https://phabricator.wikimedia.org/P80508 and previous config saved to /var/cache/conftool/dbconfig/20250802-141256-ladsgroup.json [production]
13:57 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1212 (T400854)', diff saved to https://phabricator.wikimedia.org/P80507 and previous config saved to /var/cache/conftool/dbconfig/20250802-135748-ladsgroup.json [production]
13:52 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1212 (T400854)', diff saved to https://phabricator.wikimedia.org/P80506 and previous config saved to /var/cache/conftool/dbconfig/20250802-135234-ladsgroup.json [production]
13:52 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1013,1017].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
13:52 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1212.eqiad.wmnet with reason: Maintenance [production]
13:51 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1198 (T400854)', diff saved to https://phabricator.wikimedia.org/P80505 and previous config saved to /var/cache/conftool/dbconfig/20250802-135152-ladsgroup.json [production]
13:36 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1198', diff saved to https://phabricator.wikimedia.org/P80504 and previous config saved to /var/cache/conftool/dbconfig/20250802-133645-ladsgroup.json [production]
13:21 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1198', diff saved to https://phabricator.wikimedia.org/P80503 and previous config saved to /var/cache/conftool/dbconfig/20250802-132137-ladsgroup.json [production]
13:06 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1198 (T400854)', diff saved to https://phabricator.wikimedia.org/P80502 and previous config saved to /var/cache/conftool/dbconfig/20250802-130629-ladsgroup.json [production]
13:01 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1198 (T400854)', diff saved to https://phabricator.wikimedia.org/P80501 and previous config saved to /var/cache/conftool/dbconfig/20250802-130143-ladsgroup.json [production]
13:01 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1198.eqiad.wmnet with reason: Maintenance [production]
13:01 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1189 (T400854)', diff saved to https://phabricator.wikimedia.org/P80500 and previous config saved to /var/cache/conftool/dbconfig/20250802-130120-ladsgroup.json [production]
12:46 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1189', diff saved to https://phabricator.wikimedia.org/P80499 and previous config saved to /var/cache/conftool/dbconfig/20250802-124612-ladsgroup.json [production]
12:31 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1189', diff saved to https://phabricator.wikimedia.org/P80498 and previous config saved to /var/cache/conftool/dbconfig/20250802-123105-ladsgroup.json [production]
12:15 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1189 (T400854)', diff saved to https://phabricator.wikimedia.org/P80497 and previous config saved to /var/cache/conftool/dbconfig/20250802-121557-ladsgroup.json [production]
12:11 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1189 (T400854)', diff saved to https://phabricator.wikimedia.org/P80496 and previous config saved to /var/cache/conftool/dbconfig/20250802-121112-ladsgroup.json [production]
12:11 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1189.eqiad.wmnet with reason: Maintenance [production]
12:10 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1175 (T400854)', diff saved to https://phabricator.wikimedia.org/P80495 and previous config saved to /var/cache/conftool/dbconfig/20250802-121050-ladsgroup.json [production]
11:55 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P80494 and previous config saved to /var/cache/conftool/dbconfig/20250802-115542-ladsgroup.json [production]
11:40 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P80493 and previous config saved to /var/cache/conftool/dbconfig/20250802-114035-ladsgroup.json [production]
11:25 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1175 (T400854)', diff saved to https://phabricator.wikimedia.org/P80492 and previous config saved to /var/cache/conftool/dbconfig/20250802-112527-ladsgroup.json [production]
11:20 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1175 (T400854)', diff saved to https://phabricator.wikimedia.org/P80491 and previous config saved to /var/cache/conftool/dbconfig/20250802-112037-ladsgroup.json [production]
11:20 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1175.eqiad.wmnet with reason: Maintenance [production]
11:20 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T400854)', diff saved to https://phabricator.wikimedia.org/P80490 and previous config saved to /var/cache/conftool/dbconfig/20250802-112015-ladsgroup.json [production]
11:05 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P80489 and previous config saved to /var/cache/conftool/dbconfig/20250802-110507-ladsgroup.json [production]
10:50 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P80488 and previous config saved to /var/cache/conftool/dbconfig/20250802-104959-ladsgroup.json [production]