201-250 of 10000 results (14ms)
2026-03-02 ยง
17:07 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1184', diff saved to https://phabricator.wikimedia.org/P89541 and previous config saved to /var/cache/conftool/dbconfig/20260302-170748-marostegui.json [production]
17:07 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer [toolsbeta]
17:03 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2146 (T418465)', diff saved to https://phabricator.wikimedia.org/P89540 and previous config saved to /var/cache/conftool/dbconfig/20260302-170331-marostegui.json [production]
16:52 <fceratto@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2230.codfw.wmnet with OS trixie [production]
16:52 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1184 (T418465)', diff saved to https://phabricator.wikimedia.org/P89539 and previous config saved to /var/cache/conftool/dbconfig/20260302-165240-marostegui.json [production]
16:51 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db2146 (T418465)', diff saved to https://phabricator.wikimedia.org/P89538 and previous config saved to /var/cache/conftool/dbconfig/20260302-165153-marostegui.json [production]
16:51 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2146.codfw.wmnet with reason: Maintenance [production]
16:51 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2145 (T418465)', diff saved to https://phabricator.wikimedia.org/P89537 and previous config saved to /var/cache/conftool/dbconfig/20260302-165129-marostegui.json [production]
16:46 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission [tools]
16:41 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db1184 (T418465)', diff saved to https://phabricator.wikimedia.org/P89536 and previous config saved to /var/cache/conftool/dbconfig/20260302-164141-marostegui.json [production]
16:41 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1184.eqiad.wmnet with reason: Maintenance [production]
16:41 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1169 (T418465)', diff saved to https://phabricator.wikimedia.org/P89535 and previous config saved to /var/cache/conftool/dbconfig/20260302-164118-marostegui.json [production]
16:36 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2145', diff saved to https://phabricator.wikimedia.org/P89534 and previous config saved to /var/cache/conftool/dbconfig/20260302-163622-marostegui.json [production]
16:36 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component registry-admission [tools]
16:31 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission [toolsbeta]
16:29 <fceratto@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2230.codfw.wmnet with reason: host reimage [production]
16:26 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1169', diff saved to https://phabricator.wikimedia.org/P89533 and previous config saved to /var/cache/conftool/dbconfig/20260302-162610-marostegui.json [production]
16:22 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component registry-admission [toolsbeta]
16:21 <fceratto@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on db2230.codfw.wmnet with reason: host reimage [production]
16:21 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission [tools]
16:21 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2145', diff saved to https://phabricator.wikimedia.org/P89532 and previous config saved to /var/cache/conftool/dbconfig/20260302-162115-marostegui.json [production]
16:19 <moritzm> installing PAM security updates on Bookworm [production]
16:18 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component volume-admission [tools]
16:17 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission [toolsbeta]
16:14 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component volume-admission [toolsbeta]
16:11 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1169', diff saved to https://phabricator.wikimedia.org/P89531 and previous config saved to /var/cache/conftool/dbconfig/20260302-161102-marostegui.json [production]
16:06 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2145 (T418465)', diff saved to https://phabricator.wikimedia.org/P89530 and previous config saved to /var/cache/conftool/dbconfig/20260302-160607-marostegui.json [production]
16:05 <fceratto@cumin1003> START - Cookbook sre.hosts.reimage for host db2230.codfw.wmnet with OS trixie [production]
15:56 <moritzm> installing glibc bugfix updates from trixie point release [production]
15:55 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1169 (T418465)', diff saved to https://phabricator.wikimedia.org/P89529 and previous config saved to /var/cache/conftool/dbconfig/20260302-155555-marostegui.json [production]
15:55 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db2145 (T418465)', diff saved to https://phabricator.wikimedia.org/P89528 and previous config saved to /var/cache/conftool/dbconfig/20260302-155527-marostegui.json [production]
15:55 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2145.codfw.wmnet with reason: Maintenance [production]
15:46 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2141.codfw.wmnet with reason: Maintenance [production]
15:45 <btullis@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host an-worker1169.eqiad.wmnet [production]
15:45 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db1169 (T418465)', diff saved to https://phabricator.wikimedia.org/P89527 and previous config saved to /var/cache/conftool/dbconfig/20260302-154520-marostegui.json [production]
15:45 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1169.eqiad.wmnet with reason: Maintenance [production]
15:38 <kevinbazira@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'edit-check' for release 'main' . [production]
15:34 <btullis@cumin1003> START - Cookbook sre.hosts.reboot-single for host an-worker1169.eqiad.wmnet [production]
15:33 <btullis@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host an-worker1167.eqiad.wmnet [production]
15:32 <kevinbazira@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . [production]
15:32 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2165.codfw.wmnet with reason: Maintenance [production]
15:31 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1209.eqiad.wmnet with reason: Maintenance [production]
15:31 <marostegui@cumin1003> dbctl commit (dc=all): 'Restore db1226 full weight after schema change', diff saved to https://phabricator.wikimedia.org/P89526 and previous config saved to /var/cache/conftool/dbconfig/20260302-153100-marostegui.json [production]
15:23 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1226', diff saved to https://phabricator.wikimedia.org/P89525 and previous config saved to /var/cache/conftool/dbconfig/20260302-152334-marostegui.json [production]
15:22 <btullis@cumin1003> START - Cookbook sre.hosts.reboot-single for host an-worker1167.eqiad.wmnet [production]
15:22 <btullis@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host an-worker1166.eqiad.wmnet [production]
15:18 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2198.codfw.wmnet with reason: Maintenance [production]
15:18 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2195 (T418465)', diff saved to https://phabricator.wikimedia.org/P89524 and previous config saved to /var/cache/conftool/dbconfig/20260302-151838-marostegui.json [production]
15:13 <ejegg> donorwiki upgraded from f5d7179a to 7329b41d [fundraising]
15:10 <btullis@cumin1003> START - Cookbook sre.hosts.reboot-single for host an-worker1166.eqiad.wmnet [production]