1-50 of 10000 results (108ms)
2026-05-25 §
08:52 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2231', diff saved to https://phabricator.wikimedia.org/P92845 and previous config saved to /var/cache/conftool/dbconfig/20260525-085247-fceratto.json [production]
08:42 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2231 (T426633)', diff saved to https://phabricator.wikimedia.org/P92844 and previous config saved to /var/cache/conftool/dbconfig/20260525-084239-fceratto.json [production]
08:35 <fceratto@cumin1003> dbctl commit (dc=all): 'Depooling db2231 (T426633)', diff saved to https://phabricator.wikimedia.org/P92843 and previous config saved to /var/cache/conftool/dbconfig/20260525-083540-fceratto.json [production]
08:35 <fceratto@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2231.codfw.wmnet with reason: Maintenance [production]
08:35 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2215 (T426633)', diff saved to https://phabricator.wikimedia.org/P92842 and previous config saved to /var/cache/conftool/dbconfig/20260525-083511-fceratto.json [production]
08:25 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2215', diff saved to https://phabricator.wikimedia.org/P92841 and previous config saved to /var/cache/conftool/dbconfig/20260525-082504-fceratto.json [production]
08:14 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2215', diff saved to https://phabricator.wikimedia.org/P92840 and previous config saved to /var/cache/conftool/dbconfig/20260525-081456-fceratto.json [production]
08:04 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2215 (T426633)', diff saved to https://phabricator.wikimedia.org/P92839 and previous config saved to /var/cache/conftool/dbconfig/20260525-080448-fceratto.json [production]
07:57 <fceratto@cumin1003> dbctl commit (dc=all): 'Depooling db2215 (T426633)', diff saved to https://phabricator.wikimedia.org/P92838 and previous config saved to /var/cache/conftool/dbconfig/20260525-075739-fceratto.json [production]
07:57 <fceratto@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2215.codfw.wmnet with reason: Maintenance [production]
07:57 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2196 (T426633)', diff saved to https://phabricator.wikimedia.org/P92837 and previous config saved to /var/cache/conftool/dbconfig/20260525-075708-fceratto.json [production]
07:47 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2196', diff saved to https://phabricator.wikimedia.org/P92836 and previous config saved to /var/cache/conftool/dbconfig/20260525-074700-fceratto.json [production]
07:36 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2196', diff saved to https://phabricator.wikimedia.org/P92835 and previous config saved to /var/cache/conftool/dbconfig/20260525-073653-fceratto.json [production]
07:26 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2196 (T426633)', diff saved to https://phabricator.wikimedia.org/P92834 and previous config saved to /var/cache/conftool/dbconfig/20260525-072645-fceratto.json [production]
07:19 <fceratto@cumin1003> dbctl commit (dc=all): 'Depooling db2196 (T426633)', diff saved to https://phabricator.wikimedia.org/P92833 and previous config saved to /var/cache/conftool/dbconfig/20260525-071953-fceratto.json [production]
07:19 <fceratto@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2196.codfw.wmnet with reason: Maintenance [production]
07:19 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2186 (T426633)', diff saved to https://phabricator.wikimedia.org/P92832 and previous config saved to /var/cache/conftool/dbconfig/20260525-071924-fceratto.json [production]
07:09 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2186', diff saved to https://phabricator.wikimedia.org/P92831 and previous config saved to /var/cache/conftool/dbconfig/20260525-070917-fceratto.json [production]
07:03 <marostegui@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2233.codfw.wmnet with OS trixie [production]
06:59 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2186', diff saved to https://phabricator.wikimedia.org/P92830 and previous config saved to /var/cache/conftool/dbconfig/20260525-065909-fceratto.json [production]
06:49 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2186 (T426633)', diff saved to https://phabricator.wikimedia.org/P92829 and previous config saved to /var/cache/conftool/dbconfig/20260525-064902-fceratto.json [production]
06:43 <fceratto@cumin1003> dbctl commit (dc=all): 'Depooling db2186 (T426633)', diff saved to https://phabricator.wikimedia.org/P92828 and previous config saved to /var/cache/conftool/dbconfig/20260525-064305-fceratto.json [production]
06:42 <fceratto@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]
06:40 <marostegui@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2233.codfw.wmnet with reason: host reimage [production]
06:35 <marostegui@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on db2233.codfw.wmnet with reason: host reimage [production]
06:19 <marostegui@cumin1003> START - Cookbook sre.hosts.reimage for host db2233.codfw.wmnet with OS trixie [production]
06:17 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2233.codfw.wmnet with reason: Reimage to Trixie [production]
06:17 <marostegui@cumin1003> END (FAIL) - Cookbook sre.mysql.major-upgrade (exit_code=99) [production]
06:17 <marostegui@cumin1003> START - Cookbook sre.mysql.major-upgrade [production]
06:15 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2160.codfw.wmnet with reason: Reboot upgrade m2 [production]
06:15 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2233.codfw.wmnet with reason: Reboot upgrade m2 [production]
06:08 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on dbproxy1027.eqiad.wmnet with reason: Reboot [production]
05:18 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on pc2023.codfw.wmnet,pc[1013,1023].eqiad.wmnet with reason: Maintenance on pc3 [production]
05:17 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool pc1013.eqiad.wmnet: Maintenance on pc3 [production]
05:17 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) [production]
05:17 <marostegui@cumin1003> START - Cookbook sre.mysql.parsercache [production]
05:17 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool pc1013.eqiad.wmnet: Maintenance on pc3 [production]
02:07 <mwpresync@deploy1003> Finished scap build-images: Publishing wmf/next image (duration: 06m 43s) [production]
02:00 <mwpresync@deploy1003> Started scap build-images: Publishing wmf/next image [production]
2026-05-24 §
19:08 <sukhe@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on cp6015.drmrs.wmnet with reason: hardware down [production]
02:06 <mwpresync@deploy1003> Finished scap build-images: Publishing wmf/next image (duration: 06m 23s) [production]
02:00 <mwpresync@deploy1003> Started scap build-images: Publishing wmf/next image [production]
2026-05-23 §
02:07 <mwpresync@deploy1003> Finished scap build-images: Publishing wmf/next image (duration: 06m 35s) [production]
02:01 <mwpresync@deploy1003> Started scap build-images: Publishing wmf/next image [production]
2026-05-22 §
23:39 <arlolra@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply [production]
23:39 <arlolra@deploy1003> helmfile [codfw] START helmfile.d/services/mw-parsoid: apply [production]
23:39 <arlolra@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply [production]
23:39 <arlolra@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply [production]
23:38 <arlolra@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply [production]
23:37 <arlolra@deploy1003> helmfile [codfw] START helmfile.d/services/mw-parsoid: apply [production]