4301-4350 of 10000 results (116ms)
2024-05-08 ยง
11:57 <moritzm> installing tomcat security updates [production]
11:56 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1203 (T361627)', diff saved to https://phabricator.wikimedia.org/P62074 and previous config saved to /var/cache/conftool/dbconfig/20240508-115616-marostegui.json [production]
11:37 <hnowlan> running homer commit for new codfw appservers [production]
11:30 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1203 (T361627)', diff saved to https://phabricator.wikimedia.org/P62073 and previous config saved to /var/cache/conftool/dbconfig/20240508-113048-marostegui.json [production]
11:30 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1203.eqiad.wmnet with reason: Maintenance [production]
11:30 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db1203.eqiad.wmnet with reason: Maintenance [production]
11:30 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1193 (T361627)', diff saved to https://phabricator.wikimedia.org/P62072 and previous config saved to /var/cache/conftool/dbconfig/20240508-113025-marostegui.json [production]
11:24 <marostegui@cumin1002> dbctl commit (dc=all): 'es1022 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P62071 and previous config saved to /var/cache/conftool/dbconfig/20240508-112439-root.json [production]
11:20 <marostegui@cumin1002> dbctl commit (dc=all): 'db1177 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P62070 and previous config saved to /var/cache/conftool/dbconfig/20240508-112054-root.json [production]
11:17 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host snapshot1015.eqiad.wmnet [production]
11:15 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P62069 and previous config saved to /var/cache/conftool/dbconfig/20240508-111518-marostegui.json [production]
11:10 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-host for host snapshot1015.eqiad.wmnet [production]
11:09 <hnowlan@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw2397.codfw.wmnet with OS bullseye [production]
11:09 <marostegui@cumin1002> dbctl commit (dc=all): 'es1022 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P62068 and previous config saved to /var/cache/conftool/dbconfig/20240508-110933-root.json [production]
11:08 <volans@cumin1002> END (PASS) - Cookbook sre.puppet.renew-cert (exit_code=0) for sretest1003.eqiad.wmnet: Renew puppet certificate - volans@cumin1002 [production]
11:06 <volans@cumin1002> START - Cookbook sre.puppet.renew-cert for sretest1003.eqiad.wmnet: Renew puppet certificate - volans@cumin1002 [production]
11:06 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host snapshot1011.eqiad.wmnet [production]
11:06 <volans@cumin1002> END (PASS) - Cookbook sre.puppet.renew-cert (exit_code=0) for sretest1002.eqiad.wmnet: Renew puppet certificate - volans@cumin1002 [production]
11:05 <marostegui@cumin1002> dbctl commit (dc=all): 'db1177 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P62067 and previous config saved to /var/cache/conftool/dbconfig/20240508-110545-root.json [production]
11:05 <hnowlan@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw2399.codfw.wmnet with OS bullseye [production]
11:03 <volans@cumin1002> START - Cookbook sre.puppet.renew-cert for sretest1002.eqiad.wmnet: Renew puppet certificate - volans@cumin1002 [production]
11:02 <hnowlan@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw2402.codfw.wmnet with OS bullseye [production]
11:00 <hnowlan@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw2398.codfw.wmnet with OS bullseye [production]
11:00 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P62066 and previous config saved to /var/cache/conftool/dbconfig/20240508-110010-marostegui.json [production]
10:59 <volans@cumin1002> END (PASS) - Cookbook sre.puppet.renew-cert (exit_code=0) for sretest1001.eqiad.wmnet: Renew puppet certificate - volans@cumin1002 [production]
10:57 <hnowlan@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw2401.codfw.wmnet with OS bullseye [production]
10:57 <volans@cumin1002> START - Cookbook sre.puppet.renew-cert for sretest1001.eqiad.wmnet: Renew puppet certificate - volans@cumin1002 [production]
10:55 <hnowlan@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw2396.codfw.wmnet with OS bullseye [production]
10:54 <marostegui@cumin1002> dbctl commit (dc=all): 'es1022 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P62065 and previous config saved to /var/cache/conftool/dbconfig/20240508-105428-root.json [production]
10:53 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-host for host snapshot1011.eqiad.wmnet [production]
10:50 <marostegui@cumin1002> dbctl commit (dc=all): 'db1177 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P62064 and previous config saved to /var/cache/conftool/dbconfig/20240508-105039-root.json [production]
10:50 <hnowlan@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2397.codfw.wmnet with reason: host reimage [production]
10:49 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host db2220.codfw.wmnet [production]
10:48 <ladsgroup@deploy1002> Finished scap: Backport for [[gerrit:1028952|pager: Use SelectQueryBuilder::rawTables in IndexPager (T364428)]] (duration: 15m 42s) [production]
10:46 <hnowlan@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2399.codfw.wmnet with reason: host reimage [production]
10:45 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1193 (T361627)', diff saved to https://phabricator.wikimedia.org/P62063 and previous config saved to /var/cache/conftool/dbconfig/20240508-104503-marostegui.json [production]
10:44 <hnowlan@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2402.codfw.wmnet with reason: host reimage [production]
10:41 <hnowlan@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2398.codfw.wmnet with reason: host reimage [production]
10:39 <marostegui@cumin1002> dbctl commit (dc=all): 'es1022 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P62062 and previous config saved to /var/cache/conftool/dbconfig/20240508-103922-root.json [production]
10:39 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-host for host db2220.codfw.wmnet [production]
10:38 <hnowlan@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2401.codfw.wmnet with reason: host reimage [production]
10:36 <hnowlan@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2396.codfw.wmnet with reason: host reimage [production]
10:36 <ladsgroup@deploy1002> ladsgroup: Continuing with sync [production]
10:35 <ladsgroup@deploy1002> ladsgroup: Backport for [[gerrit:1028952|pager: Use SelectQueryBuilder::rawTables in IndexPager (T364428)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
10:35 <marostegui@cumin1002> dbctl commit (dc=all): 'db1177 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P62061 and previous config saved to /var/cache/conftool/dbconfig/20240508-103531-root.json [production]
10:34 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host db2218.codfw.wmnet [production]
10:34 <marostegui@cumin1002> dbctl commit (dc=all): 'es2023 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P62060 and previous config saved to /var/cache/conftool/dbconfig/20240508-103410-root.json [production]
10:33 <hnowlan@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw2398.codfw.wmnet with reason: host reimage [production]
10:33 <hnowlan@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw2397.codfw.wmnet with reason: host reimage [production]
10:33 <ladsgroup@deploy1002> Started scap: Backport for [[gerrit:1028952|pager: Use SelectQueryBuilder::rawTables in IndexPager (T364428)]] [production]