4101-4150 of 10000 results (92ms)
2024-02-20 ยง
17:59 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2171 (T357189)', diff saved to https://phabricator.wikimedia.org/P57398 and previous config saved to /var/cache/conftool/dbconfig/20240220-175917-arnaudb.json [production]
17:56 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1219 (T355609)', diff saved to https://phabricator.wikimedia.org/P57397 and previous config saved to /var/cache/conftool/dbconfig/20240220-175605-marostegui.json [production]
17:44 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2171', diff saved to https://phabricator.wikimedia.org/P57396 and previous config saved to /var/cache/conftool/dbconfig/20240220-174411-arnaudb.json [production]
17:29 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2171', diff saved to https://phabricator.wikimedia.org/P57395 and previous config saved to /var/cache/conftool/dbconfig/20240220-172904-arnaudb.json [production]
17:27 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1219 (T355609)', diff saved to https://phabricator.wikimedia.org/P57394 and previous config saved to /var/cache/conftool/dbconfig/20240220-172716-marostegui.json [production]
17:27 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1219.eqiad.wmnet with reason: Maintenance [production]
17:27 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 6:00:00 on db1219.eqiad.wmnet with reason: Maintenance [production]
17:26 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1218 (T355609)', diff saved to https://phabricator.wikimedia.org/P57393 and previous config saved to /var/cache/conftool/dbconfig/20240220-172653-marostegui.json [production]
17:18 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp4052.ulsfo.wmnet with OS bookworm [production]
17:13 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2171 (T357189)', diff saved to https://phabricator.wikimedia.org/P57392 and previous config saved to /var/cache/conftool/dbconfig/20240220-171358-arnaudb.json [production]
17:11 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1218', diff saved to https://phabricator.wikimedia.org/P57391 and previous config saved to /var/cache/conftool/dbconfig/20240220-171147-marostegui.json [production]
17:10 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db2171 (T357189)', diff saved to https://phabricator.wikimedia.org/P57390 and previous config saved to /var/cache/conftool/dbconfig/20240220-170949-arnaudb.json [production]
17:09 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2171.codfw.wmnet with reason: Maintenance [production]
17:09 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2171.codfw.wmnet with reason: Maintenance [production]
17:09 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2157 (T357189)', diff saved to https://phabricator.wikimedia.org/P57389 and previous config saved to /var/cache/conftool/dbconfig/20240220-170928-arnaudb.json [production]
16:57 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp4052.ulsfo.wmnet with reason: host reimage [production]
16:56 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1218', diff saved to https://phabricator.wikimedia.org/P57388 and previous config saved to /var/cache/conftool/dbconfig/20240220-165641-marostegui.json [production]
16:55 <sukhe@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cp4052.ulsfo.wmnet with reason: host reimage [production]
16:54 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2157', diff saved to https://phabricator.wikimedia.org/P57387 and previous config saved to /var/cache/conftool/dbconfig/20240220-165421-arnaudb.json [production]
16:43 <ayounsi@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host sretest2005.codfw.wmnet with OS bookworm [production]
16:43 <brett@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=codfw,name=cp20(29|30).codfw.wmnet [production]
16:42 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for cp[2029-2030].codfw.wmnet [production]
16:42 <brett@cumin2002> START - Cookbook sre.hosts.remove-downtime for cp[2029-2030].codfw.wmnet [production]
16:41 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1218 (T355609)', diff saved to https://phabricator.wikimedia.org/P57386 and previous config saved to /var/cache/conftool/dbconfig/20240220-164134-marostegui.json [production]
16:39 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2157', diff saved to https://phabricator.wikimedia.org/P57385 and previous config saved to /var/cache/conftool/dbconfig/20240220-163915-arnaudb.json [production]
16:35 <reedy@deploy2002> Synchronized php-1.42.0-wmf.19/extensions/AntiSpoof/: T357995 (duration: 11m 02s) [production]
16:35 <arnaudb@cumin1002> dbctl commit (dc=all): 'db1233 (re)pooling @ 100%: Maintenance done', diff saved to https://phabricator.wikimedia.org/P57384 and previous config saved to /var/cache/conftool/dbconfig/20240220-163451-arnaudb.json [production]
16:35 <arnaudb@cumin1002> dbctl commit (dc=all): 'db1210 (re)pooling @ 100%: Maintenance done', diff saved to https://phabricator.wikimedia.org/P57383 and previous config saved to /var/cache/conftool/dbconfig/20240220-163447-arnaudb.json [production]
16:34 <arnaudb@cumin1002> dbctl commit (dc=all): 'db1168 (re)pooling @ 100%: Maintenance done', diff saved to https://phabricator.wikimedia.org/P57382 and previous config saved to /var/cache/conftool/dbconfig/20240220-163447-arnaudb.json [production]
16:34 <arnaudb@cumin1002> dbctl commit (dc=all): 'db1226 (re)pooling @ 100%: maintenance done', diff saved to https://phabricator.wikimedia.org/P57381 and previous config saved to /var/cache/conftool/dbconfig/20240220-163442-arnaudb.json [production]
16:30 <fnegri@cumin1002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudrabbit1001.eqiad.wmnet [production]
16:29 <sukhe@cumin2002> START - Cookbook sre.hosts.reimage for host cp4052.ulsfo.wmnet with OS bookworm [production]
16:27 <sukhe@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp4052.ulsfo.wmnet with OS bookworm [production]
16:24 <fnegri@cumin1002> START - Cookbook sre.hosts.reboot-single for host cloudrabbit1001.eqiad.wmnet [production]
16:24 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2157 (T357189)', diff saved to https://phabricator.wikimedia.org/P57380 and previous config saved to /var/cache/conftool/dbconfig/20240220-162408-arnaudb.json [production]
16:21 <fnegri@cumin1002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudrabbit1002.eqiad.wmnet [production]
16:20 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db2157 (T357189)', diff saved to https://phabricator.wikimedia.org/P57379 and previous config saved to /var/cache/conftool/dbconfig/20240220-161953-arnaudb.json [production]
16:20 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2157.codfw.wmnet with reason: Maintenance [production]
16:20 <arnaudb@cumin1002> dbctl commit (dc=all): 'db1233 (re)pooling @ 75%: Maintenance done', diff saved to https://phabricator.wikimedia.org/P57378 and previous config saved to /var/cache/conftool/dbconfig/20240220-161946-arnaudb.json [production]
16:20 <arnaudb@cumin1002> dbctl commit (dc=all): 'db1210 (re)pooling @ 75%: Maintenance done', diff saved to https://phabricator.wikimedia.org/P57377 and previous config saved to /var/cache/conftool/dbconfig/20240220-161942-arnaudb.json [production]
16:19 <arnaudb@cumin1002> dbctl commit (dc=all): 'db1168 (re)pooling @ 75%: Maintenance done', diff saved to https://phabricator.wikimedia.org/P57376 and previous config saved to /var/cache/conftool/dbconfig/20240220-161942-arnaudb.json [production]
16:19 <arnaudb@cumin1002> dbctl commit (dc=all): 'db1226 (re)pooling @ 75%: maintenance done', diff saved to https://phabricator.wikimedia.org/P57375 and previous config saved to /var/cache/conftool/dbconfig/20240220-161937-arnaudb.json [production]
16:19 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2157.codfw.wmnet with reason: Maintenance [production]
16:19 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2128 (T357189)', diff saved to https://phabricator.wikimedia.org/P57374 and previous config saved to /var/cache/conftool/dbconfig/20240220-161931-arnaudb.json [production]
16:18 <sukhe@cumin2002> START - Cookbook sre.hosts.reimage for host cp4052.ulsfo.wmnet with OS bookworm [production]
16:14 <fnegri@cumin1002> START - Cookbook sre.hosts.reboot-single for host cloudrabbit1002.eqiad.wmnet [production]
16:13 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1218 (T355609)', diff saved to https://phabricator.wikimedia.org/P57373 and previous config saved to /var/cache/conftool/dbconfig/20240220-161348-marostegui.json [production]
16:13 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1218.eqiad.wmnet with reason: Maintenance [production]
16:13 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 6:00:00 on db1218.eqiad.wmnet with reason: Maintenance [production]
16:13 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1207 (T355609)', diff saved to https://phabricator.wikimedia.org/P57372 and previous config saved to /var/cache/conftool/dbconfig/20240220-161326-marostegui.json [production]