|
2025-01-28
ยง
|
| 10:05 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1170', diff saved to https://phabricator.wikimedia.org/P72587 and previous config saved to /var/cache/conftool/dbconfig/20250128-100539-marostegui.json |
[production] |
| 10:04 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db2205 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P72586 and previous config saved to /var/cache/conftool/dbconfig/20250128-100434-root.json |
[production] |
| 10:04 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1175 (re)pooling @ 75%: Repooling T384807', diff saved to https://phabricator.wikimedia.org/P72585 and previous config saved to /var/cache/conftool/dbconfig/20250128-100402-root.json |
[production] |
| 09:58 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'es2036 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P72584 and previous config saved to /var/cache/conftool/dbconfig/20250128-095818-root.json |
[production] |
| 09:50 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1170 (T384592)', diff saved to https://phabricator.wikimedia.org/P72583 and previous config saved to /var/cache/conftool/dbconfig/20250128-095032-marostegui.json |
[production] |
| 09:50 |
<root@cumin1002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1150.eqiad.wmnet with OS bookworm |
[production] |
| 09:49 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db2205 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P72582 and previous config saved to /var/cache/conftool/dbconfig/20250128-094928-root.json |
[production] |
| 09:48 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1175 (re)pooling @ 50%: Repooling T384807', diff saved to https://phabricator.wikimedia.org/P72581 and previous config saved to /var/cache/conftool/dbconfig/20250128-094857-root.json |
[production] |
| 09:43 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'es2036 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P72580 and previous config saved to /var/cache/conftool/dbconfig/20250128-094313-root.json |
[production] |
| 09:41 |
<root@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2190.codfw.wmnet with reason: Index rebuild |
[production] |
| 09:39 |
<root@cumin1002> |
END (PASS) - Cookbook sre.mysql.upgrade (exit_code=0) for db2190.codfw.wmnet |
[production] |
| 09:34 |
<root@cumin1002> |
START - Cookbook sre.mysql.upgrade for db2190.codfw.wmnet |
[production] |
| 09:34 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db2190', diff saved to https://phabricator.wikimedia.org/P72578 and previous config saved to /var/cache/conftool/dbconfig/20250128-093446-marostegui.json |
[production] |
| 09:34 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db2205 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P72577 and previous config saved to /var/cache/conftool/dbconfig/20250128-093423-root.json |
[production] |
| 09:33 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1175 (re)pooling @ 25%: Repooling T384807', diff saved to https://phabricator.wikimedia.org/P72576 and previous config saved to /var/cache/conftool/dbconfig/20250128-093352-root.json |
[production] |
| 09:28 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'es2036 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P72575 and previous config saved to /var/cache/conftool/dbconfig/20250128-092808-root.json |
[production] |
| 09:27 |
<fabfur> |
installing/enabling haproxykafka on codfw (https://gerrit.wikimedia.org/r/c/operations/puppet/+/1114415) (T378578) |
[production] |
| 09:26 |
<root@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1150.eqiad.wmnet with reason: host reimage |
[production] |
| 09:22 |
<root@cumin1002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on db1150.eqiad.wmnet with reason: host reimage |
[production] |
| 09:22 |
<root@cumin1002> |
END (PASS) - Cookbook sre.mysql.upgrade (exit_code=0) for es2030.codfw.wmnet |
[production] |
| 09:18 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1175 (re)pooling @ 10%: Repooling T384807', diff saved to https://phabricator.wikimedia.org/P72574 and previous config saved to /var/cache/conftool/dbconfig/20250128-091846-root.json |
[production] |
| 09:13 |
<root@cumin1002> |
START - Cookbook sre.mysql.upgrade for es2030.codfw.wmnet |
[production] |
| 09:13 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'es2036 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P72572 and previous config saved to /var/cache/conftool/dbconfig/20250128-091302-root.json |
[production] |
| 09:12 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool es2030', diff saved to https://phabricator.wikimedia.org/P72571 and previous config saved to /var/cache/conftool/dbconfig/20250128-091242-marostegui.json |
[production] |
| 09:06 |
<root@cumin1002> |
START - Cookbook sre.hosts.reimage for host db1150.eqiad.wmnet with OS bookworm |
[production] |
| 09:06 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db1175', diff saved to https://phabricator.wikimedia.org/P72570 and previous config saved to /var/cache/conftool/dbconfig/20250128-090620-marostegui.json |
[production] |
| 09:06 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db2205', diff saved to https://phabricator.wikimedia.org/P72569 and previous config saved to /var/cache/conftool/dbconfig/20250128-090601-marostegui.json |
[production] |
| 09:05 |
<root@cumin1002> |
END (PASS) - Cookbook sre.mysql.upgrade (exit_code=0) for es2036.codfw.wmnet |
[production] |
| 09:05 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2020.codfw.wmnet |
[production] |
| 09:05 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db2205 (re)pooling @ 25%: 10', diff saved to https://phabricator.wikimedia.org/P72568 and previous config saved to /var/cache/conftool/dbconfig/20250128-090525-root.json |
[production] |
| 09:04 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Promote es2028 to es1 codfw master', diff saved to https://phabricator.wikimedia.org/P72567 and previous config saved to /var/cache/conftool/dbconfig/20250128-090454-marostegui.json |
[production] |
| 09:04 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1175 (re)pooling @ 25%: 10', diff saved to https://phabricator.wikimedia.org/P72566 and previous config saved to /var/cache/conftool/dbconfig/20250128-090439-root.json |
[production] |
| 09:00 |
<jynus@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db1150.eqiad.wmnet with reason: reimage |
[production] |
| 09:00 |
<root@cumin1002> |
START - Cookbook sre.mysql.upgrade for es2036.codfw.wmnet |
[production] |
| 09:00 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool es2036', diff saved to https://phabricator.wikimedia.org/P72565 and previous config saved to /var/cache/conftool/dbconfig/20250128-090000-marostegui.json |
[production] |
| 08:57 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2020.codfw.wmnet |
[production] |
| 08:56 |
<reedy@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1110053|CommonSettings: Set 'lang=en' on Wikimedia Foundation entry in $wgFooterIcons (T383501)]], [[gerrit:1114454|SimpleCaptcha: Don't look up captcha if no ID was given (T384858)]], [[gerrit:1091818|noc: Expose MobileUrlCallback.php]], [[gerrit:1108179|Disable Dns Blacklist checks (T382987)]], [[gerrit:1088655|CommonSettings.php: Remove deprecated $w |
[production] |
| 08:55 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1170 (T384592)', diff saved to https://phabricator.wikimedia.org/P72564 and previous config saved to /var/cache/conftool/dbconfig/20250128-085528-marostegui.json |
[production] |
| 08:55 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1170.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:55 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1158 (T384592)', diff saved to https://phabricator.wikimedia.org/P72563 and previous config saved to /var/cache/conftool/dbconfig/20250128-085506-marostegui.json |
[production] |
| 08:51 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ganeti2020.codfw.wmnet with OS bookworm |
[production] |
| 08:48 |
<reedy@deploy2002> |
reedy, jforrester: Continuing with sync |
[production] |
| 08:48 |
<reedy@deploy2002> |
reedy, jforrester: Backport for [[gerrit:1110053|CommonSettings: Set 'lang=en' on Wikimedia Foundation entry in $wgFooterIcons (T383501)]], [[gerrit:1114454|SimpleCaptcha: Don't look up captcha if no ID was given (T384858)]], [[gerrit:1091818|noc: Expose MobileUrlCallback.php]], [[gerrit:1108179|Disable Dns Blacklist checks (T382987)]], [[gerrit:1088655|CommonSettings.php: Remove deprecated $wgOATHAu |
[production] |
| 08:43 |
<reedy@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1110053|CommonSettings: Set 'lang=en' on Wikimedia Foundation entry in $wgFooterIcons (T383501)]], [[gerrit:1114454|SimpleCaptcha: Don't look up captcha if no ID was given (T384858)]], [[gerrit:1091818|noc: Expose MobileUrlCallback.php]], [[gerrit:1108179|Disable Dns Blacklist checks (T382987)]], [[gerrit:1088655|CommonSettings.php: Remove deprecated $wg |
[production] |
| 08:40 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1158', diff saved to https://phabricator.wikimedia.org/P72562 and previous config saved to /var/cache/conftool/dbconfig/20250128-083959-marostegui.json |
[production] |
| 08:33 |
<urbanecm@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1114379|[Growth] enwiki: Release Add Link to 10% of newcomers (T384551)]] (duration: 16m 46s) |
[production] |
| 08:24 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1158', diff saved to https://phabricator.wikimedia.org/P72561 and previous config saved to /var/cache/conftool/dbconfig/20250128-082452-marostegui.json |
[production] |
| 08:24 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ganeti2020.codfw.wmnet with reason: host reimage |
[production] |
| 08:24 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2026.codfw.wmnet |
[production] |
| 08:23 |
<urbanecm@deploy2002> |
urbanecm: Continuing with sync |
[production] |