|
2025-01-28
ยง
|
| 09:26 |
<root@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1150.eqiad.wmnet with reason: host reimage |
[production] |
| 09:22 |
<root@cumin1002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on db1150.eqiad.wmnet with reason: host reimage |
[production] |
| 09:22 |
<root@cumin1002> |
END (PASS) - Cookbook sre.mysql.upgrade (exit_code=0) for es2030.codfw.wmnet |
[production] |
| 09:18 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1175 (re)pooling @ 10%: Repooling T384807', diff saved to https://phabricator.wikimedia.org/P72574 and previous config saved to /var/cache/conftool/dbconfig/20250128-091846-root.json |
[production] |
| 09:13 |
<root@cumin1002> |
START - Cookbook sre.mysql.upgrade for es2030.codfw.wmnet |
[production] |
| 09:13 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'es2036 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P72572 and previous config saved to /var/cache/conftool/dbconfig/20250128-091302-root.json |
[production] |
| 09:12 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool es2030', diff saved to https://phabricator.wikimedia.org/P72571 and previous config saved to /var/cache/conftool/dbconfig/20250128-091242-marostegui.json |
[production] |
| 09:06 |
<root@cumin1002> |
START - Cookbook sre.hosts.reimage for host db1150.eqiad.wmnet with OS bookworm |
[production] |
| 09:06 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db1175', diff saved to https://phabricator.wikimedia.org/P72570 and previous config saved to /var/cache/conftool/dbconfig/20250128-090620-marostegui.json |
[production] |
| 09:06 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db2205', diff saved to https://phabricator.wikimedia.org/P72569 and previous config saved to /var/cache/conftool/dbconfig/20250128-090601-marostegui.json |
[production] |
| 09:05 |
<root@cumin1002> |
END (PASS) - Cookbook sre.mysql.upgrade (exit_code=0) for es2036.codfw.wmnet |
[production] |
| 09:05 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2020.codfw.wmnet |
[production] |
| 09:05 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db2205 (re)pooling @ 25%: 10', diff saved to https://phabricator.wikimedia.org/P72568 and previous config saved to /var/cache/conftool/dbconfig/20250128-090525-root.json |
[production] |
| 09:04 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Promote es2028 to es1 codfw master', diff saved to https://phabricator.wikimedia.org/P72567 and previous config saved to /var/cache/conftool/dbconfig/20250128-090454-marostegui.json |
[production] |
| 09:04 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1175 (re)pooling @ 25%: 10', diff saved to https://phabricator.wikimedia.org/P72566 and previous config saved to /var/cache/conftool/dbconfig/20250128-090439-root.json |
[production] |
| 09:00 |
<jynus@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db1150.eqiad.wmnet with reason: reimage |
[production] |
| 09:00 |
<root@cumin1002> |
START - Cookbook sre.mysql.upgrade for es2036.codfw.wmnet |
[production] |
| 09:00 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool es2036', diff saved to https://phabricator.wikimedia.org/P72565 and previous config saved to /var/cache/conftool/dbconfig/20250128-090000-marostegui.json |
[production] |
| 08:57 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2020.codfw.wmnet |
[production] |
| 08:56 |
<reedy@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1110053|CommonSettings: Set 'lang=en' on Wikimedia Foundation entry in $wgFooterIcons (T383501)]], [[gerrit:1114454|SimpleCaptcha: Don't look up captcha if no ID was given (T384858)]], [[gerrit:1091818|noc: Expose MobileUrlCallback.php]], [[gerrit:1108179|Disable Dns Blacklist checks (T382987)]], [[gerrit:1088655|CommonSettings.php: Remove deprecated $w |
[production] |
| 08:55 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1170 (T384592)', diff saved to https://phabricator.wikimedia.org/P72564 and previous config saved to /var/cache/conftool/dbconfig/20250128-085528-marostegui.json |
[production] |
| 08:55 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1170.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:55 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1158 (T384592)', diff saved to https://phabricator.wikimedia.org/P72563 and previous config saved to /var/cache/conftool/dbconfig/20250128-085506-marostegui.json |
[production] |
| 08:51 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ganeti2020.codfw.wmnet with OS bookworm |
[production] |
| 08:48 |
<reedy@deploy2002> |
reedy, jforrester: Continuing with sync |
[production] |
| 08:48 |
<reedy@deploy2002> |
reedy, jforrester: Backport for [[gerrit:1110053|CommonSettings: Set 'lang=en' on Wikimedia Foundation entry in $wgFooterIcons (T383501)]], [[gerrit:1114454|SimpleCaptcha: Don't look up captcha if no ID was given (T384858)]], [[gerrit:1091818|noc: Expose MobileUrlCallback.php]], [[gerrit:1108179|Disable Dns Blacklist checks (T382987)]], [[gerrit:1088655|CommonSettings.php: Remove deprecated $wgOATHAu |
[production] |
| 08:43 |
<reedy@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1110053|CommonSettings: Set 'lang=en' on Wikimedia Foundation entry in $wgFooterIcons (T383501)]], [[gerrit:1114454|SimpleCaptcha: Don't look up captcha if no ID was given (T384858)]], [[gerrit:1091818|noc: Expose MobileUrlCallback.php]], [[gerrit:1108179|Disable Dns Blacklist checks (T382987)]], [[gerrit:1088655|CommonSettings.php: Remove deprecated $wg |
[production] |
| 08:40 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1158', diff saved to https://phabricator.wikimedia.org/P72562 and previous config saved to /var/cache/conftool/dbconfig/20250128-083959-marostegui.json |
[production] |
| 08:33 |
<urbanecm@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1114379|[Growth] enwiki: Release Add Link to 10% of newcomers (T384551)]] (duration: 16m 46s) |
[production] |
| 08:24 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1158', diff saved to https://phabricator.wikimedia.org/P72561 and previous config saved to /var/cache/conftool/dbconfig/20250128-082452-marostegui.json |
[production] |
| 08:24 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ganeti2020.codfw.wmnet with reason: host reimage |
[production] |
| 08:24 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2026.codfw.wmnet |
[production] |
| 08:23 |
<urbanecm@deploy2002> |
urbanecm: Continuing with sync |
[production] |
| 08:23 |
<urbanecm@deploy2002> |
urbanecm: Backport for [[gerrit:1114379|[Growth] enwiki: Release Add Link to 10% of newcomers (T384551)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 08:21 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on ganeti2020.codfw.wmnet with reason: host reimage |
[production] |
| 08:16 |
<urbanecm@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1114379|[Growth] enwiki: Release Add Link to 10% of newcomers (T384551)]] |
[production] |
| 08:09 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1158 (T384592)', diff saved to https://phabricator.wikimedia.org/P72560 and previous config saved to /var/cache/conftool/dbconfig/20250128-080945-marostegui.json |
[production] |
| 08:07 |
<root@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2155.codfw.wmnet with reason: Index rebuild |
[production] |
| 08:06 |
<root@cumin1002> |
END (PASS) - Cookbook sre.mysql.upgrade (exit_code=0) for db2155.codfw.wmnet |
[production] |
| 08:06 |
<slyngshede@dns1004> |
END - running authdns-update |
[production] |
| 08:04 |
<slyngshede@dns1004> |
START - running authdns-update |
[production] |
| 08:00 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db[2186-2187].codfw.wmnet with reason: Index rebuild + upgrade |
[production] |
| 07:59 |
<root@cumin1002> |
START - Cookbook sre.mysql.upgrade for db2155.codfw.wmnet |
[production] |
| 07:59 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2026.codfw.wmnet |
[production] |
| 07:59 |
<slyngshede@dns1004> |
END - running authdns-update |
[production] |
| 07:58 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db2155 T382842', diff saved to https://phabricator.wikimedia.org/P72559 and previous config saved to /var/cache/conftool/dbconfig/20250128-075857-marostegui.json |
[production] |
| 07:57 |
<slyngshede@dns1004> |
START - running authdns-update |
[production] |
| 07:56 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2026.codfw.wmnet |
[production] |
| 07:56 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1158 (T384592)', diff saved to https://phabricator.wikimedia.org/P72558 and previous config saved to /var/cache/conftool/dbconfig/20250128-075636-marostegui.json |
[production] |
| 07:56 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1014,1018].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance |
[production] |