5301-5350 of 10000 results (64ms)
2022-05-30 ยง
13:14 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184 (T60674)', diff saved to https://phabricator.wikimedia.org/P28970 and previous config saved to /var/cache/conftool/dbconfig/20220530-131419-ladsgroup.json [production]
13:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1178 (T309311)', diff saved to https://phabricator.wikimedia.org/P28969 and previous config saved to /var/cache/conftool/dbconfig/20220530-131057-ladsgroup.json [production]
13:10 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1178.eqiad.wmnet with reason: Maintenance [production]
13:10 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1178.eqiad.wmnet with reason: Maintenance [production]
13:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1167 (T309311)', diff saved to https://phabricator.wikimedia.org/P28968 and previous config saved to /var/cache/conftool/dbconfig/20220530-131049-ladsgroup.json [production]
13:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1098:3317', diff saved to https://phabricator.wikimedia.org/P28967 and previous config saved to /var/cache/conftool/dbconfig/20220530-131044-ladsgroup.json [production]
13:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1138 (T60674)', diff saved to https://phabricator.wikimedia.org/P28966 and previous config saved to /var/cache/conftool/dbconfig/20220530-131003-ladsgroup.json [production]
13:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1175 (T60674)', diff saved to https://phabricator.wikimedia.org/P28965 and previous config saved to /var/cache/conftool/dbconfig/20220530-130642-ladsgroup.json [production]
13:06 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1175.eqiad.wmnet with reason: Maintenance [production]
13:06 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1175.eqiad.wmnet with reason: Maintenance [production]
13:05 <marostegui@cumin1001> dbctl commit (dc=all): 'db1111 (re)pooling @ 25%: After migrating it to 10.6', diff saved to https://phabricator.wikimedia.org/P28964 and previous config saved to /var/cache/conftool/dbconfig/20220530-130545-root.json [production]
13:03 <sukhe> upload pdns-recursor_4.6.2-1wm1 to apt.wm.o (bullseye) - T305589 [production]
13:03 <akosiaris@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mathoid: apply [production]
13:02 <akosiaris@deploy1002> helmfile [eqiad] START helmfile.d/services/mathoid: apply [production]
12:58 <akosiaris@deploy1002> helmfile [codfw] DONE helmfile.d/services/mathoid: apply [production]
12:57 <akosiaris@deploy1002> helmfile [codfw] START helmfile.d/services/mathoid: apply [production]
12:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P28963 and previous config saved to /var/cache/conftool/dbconfig/20220530-125544-ladsgroup.json [production]
12:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1098:3317 (T60674)', diff saved to https://phabricator.wikimedia.org/P28962 and previous config saved to /var/cache/conftool/dbconfig/20220530-125539-ladsgroup.json [production]
12:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1184 (T60674)', diff saved to https://phabricator.wikimedia.org/P28961 and previous config saved to /var/cache/conftool/dbconfig/20220530-125233-ladsgroup.json [production]
12:52 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1184.eqiad.wmnet with reason: Maintenance [production]
12:52 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1184.eqiad.wmnet with reason: Maintenance [production]
12:50 <marostegui@cumin1001> dbctl commit (dc=all): 'db1111 (re)pooling @ 20%: After migrating it to 10.6', diff saved to https://phabricator.wikimedia.org/P28960 and previous config saved to /var/cache/conftool/dbconfig/20220530-125042-root.json [production]
12:48 <akosiaris@deploy1002> helmfile [staging] DONE helmfile.d/services/mathoid: apply [production]
12:48 <akosiaris@deploy1002> helmfile [staging] START helmfile.d/services/mathoid: apply [production]
12:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P28959 and previous config saved to /var/cache/conftool/dbconfig/20220530-124039-ladsgroup.json [production]
12:38 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1128.eqiad.wmnet with reason: Maintenance [production]
12:38 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1128.eqiad.wmnet with reason: Maintenance [production]
12:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1138 (T60674)', diff saved to https://phabricator.wikimedia.org/P28958 and previous config saved to /var/cache/conftool/dbconfig/20220530-123716-ladsgroup.json [production]
12:37 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1138.eqiad.wmnet with reason: Maintenance [production]
12:37 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1138.eqiad.wmnet with reason: Maintenance [production]
12:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3314 (T60674)', diff saved to https://phabricator.wikimedia.org/P28957 and previous config saved to /var/cache/conftool/dbconfig/20220530-123708-ladsgroup.json [production]
12:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1098:3317 (T60674)', diff saved to https://phabricator.wikimedia.org/P28956 and previous config saved to /var/cache/conftool/dbconfig/20220530-123644-ladsgroup.json [production]
12:36 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1098.eqiad.wmnet with reason: Maintenance [production]
12:36 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1098.eqiad.wmnet with reason: Maintenance [production]
12:35 <marostegui@cumin1001> dbctl commit (dc=all): 'db1111 (re)pooling @ 10%: After migrating it to 10.6', diff saved to https://phabricator.wikimedia.org/P28955 and previous config saved to /var/cache/conftool/dbconfig/20220530-123538-root.json [production]
12:34 <marostegui> Drop renamed revision_actor_temp on s3 T307906 [production]
12:26 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on 6 hosts with reason: Maintenance [production]
12:26 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on 6 hosts with reason: Maintenance [production]
12:26 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2105.codfw.wmnet with reason: Maintenance [production]
12:26 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2105.codfw.wmnet with reason: Maintenance [production]
12:26 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112 (T60674)', diff saved to https://phabricator.wikimedia.org/P28954 and previous config saved to /var/cache/conftool/dbconfig/20220530-122625-ladsgroup.json [production]
12:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1167 (T309311)', diff saved to https://phabricator.wikimedia.org/P28953 and previous config saved to /var/cache/conftool/dbconfig/20220530-122534-ladsgroup.json [production]
12:24 <marostegui@cumin1001> dbctl commit (dc=all): 'db1184 (re)pooling @ 100%: After cloning db1128', diff saved to https://phabricator.wikimedia.org/P28952 and previous config saved to /var/cache/conftool/dbconfig/20220530-122454-root.json [production]
12:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3314', diff saved to https://phabricator.wikimedia.org/P28951 and previous config saved to /var/cache/conftool/dbconfig/20220530-122203-ladsgroup.json [production]
12:20 <marostegui@cumin1001> dbctl commit (dc=all): 'db1111 (re)pooling @ 5%: After migrating it to 10.6', diff saved to https://phabricator.wikimedia.org/P28950 and previous config saved to /var/cache/conftool/dbconfig/20220530-122034-root.json [production]
12:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112', diff saved to https://phabricator.wikimedia.org/P28949 and previous config saved to /var/cache/conftool/dbconfig/20220530-121120-ladsgroup.json [production]
12:09 <marostegui@cumin1001> dbctl commit (dc=all): 'db1184 (re)pooling @ 75%: After cloning db1128', diff saved to https://phabricator.wikimedia.org/P28948 and previous config saved to /var/cache/conftool/dbconfig/20220530-120950-root.json [production]
12:09 <moritzm> installing java 8/11 security updates [production]
12:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3314', diff saved to https://phabricator.wikimedia.org/P28947 and previous config saved to /var/cache/conftool/dbconfig/20220530-120658-ladsgroup.json [production]
12:05 <marostegui@cumin1001> dbctl commit (dc=all): 'db1111 (re)pooling @ 1%: After migrating it to 10.6', diff saved to https://phabricator.wikimedia.org/P28946 and previous config saved to /var/cache/conftool/dbconfig/20220530-120530-root.json [production]