|
2024-11-29
ยง
|
| 06:26 |
<marostegui@cumin2002> |
START - Cookbook sre.mysql.pool db1223 quickly with 2 steps - Fixed corruption |
[production] |
| 05:52 |
<taavi@cumin1002> |
dbctl commit (dc=all): 'depool db1223, replication broken', diff saved to https://phabricator.wikimedia.org/P71423 and previous config saved to /var/cache/conftool/dbconfig/20241129-055245-taavi.json |
[production] |
| 05:00 |
<raymond-ndibe@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-harbor |
[toolsbeta] |
| 05:00 |
<raymond-ndibe@cloudcumin1001> |
START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor |
[toolsbeta] |
| 04:54 |
<raymond-ndibe@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-harbor |
[toolsbeta] |
| 04:54 |
<raymond-ndibe@cloudcumin1001> |
START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor |
[toolsbeta] |
| 04:54 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2229 (T328817)', diff saved to https://phabricator.wikimedia.org/P71422 and previous config saved to /var/cache/conftool/dbconfig/20241129-045409-ladsgroup.json |
[production] |
| 04:51 |
<raymond-ndibe@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-harbor |
[toolsbeta] |
| 04:51 |
<raymond-ndibe@cloudcumin1001> |
START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor |
[toolsbeta] |
| 04:40 |
<raymond-ndibe@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-harbor |
[toolsbeta] |
| 04:40 |
<raymond-ndibe@cloudcumin1001> |
START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor |
[toolsbeta] |
| 04:39 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2229', diff saved to https://phabricator.wikimedia.org/P71421 and previous config saved to /var/cache/conftool/dbconfig/20241129-043902-ladsgroup.json |
[production] |
| 04:23 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2229', diff saved to https://phabricator.wikimedia.org/P71420 and previous config saved to /var/cache/conftool/dbconfig/20241129-042355-ladsgroup.json |
[production] |
| 04:20 |
<tstarling@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1098915|addWiki.php tweaks]], [[gerrit:1098916|Run dumpInterwiki.php locally with no changes]], [[gerrit:1098917|Prepare id.wikivoyage.org for installation (T380726 T352113)]], [[gerrit:1099065|dumpInterwiki: read from preinstall.dblist (T352113)]], [[gerrit:1099066|addWiki: Move DB_ADMIN to core]], [[gerrit:1099064|addWiki: Add UpdateSearchIndexCon |
[production] |
| 04:14 |
<raymond-ndibe@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-harbor |
[toolsbeta] |
| 04:14 |
<raymond-ndibe@cloudcumin1001> |
START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor |
[toolsbeta] |
| 04:13 |
<raymond-ndibe@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-harbor |
[toolsbeta] |
| 04:13 |
<raymond-ndibe@cloudcumin1001> |
START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor |
[toolsbeta] |
| 04:12 |
<tstarling@deploy2002> |
tstarling: Continuing with sync |
[production] |
| 04:12 |
<tstarling@deploy2002> |
tstarling: Backport for [[gerrit:1098915|addWiki.php tweaks]], [[gerrit:1098916|Run dumpInterwiki.php locally with no changes]], [[gerrit:1098917|Prepare id.wikivoyage.org for installation (T380726 T352113)]], [[gerrit:1099065|dumpInterwiki: read from preinstall.dblist (T352113)]], [[gerrit:1099066|addWiki: Move DB_ADMIN to core]], [[gerrit:1099064|addWiki: Add UpdateSearchIndexConfig]], [[gerrit |
[production] |
| 04:08 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2229 (T328817)', diff saved to https://phabricator.wikimedia.org/P71419 and previous config saved to /var/cache/conftool/dbconfig/20241129-040846-ladsgroup.json |
[production] |
| 04:05 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db2229 (T328817)', diff saved to https://phabricator.wikimedia.org/P71418 and previous config saved to /var/cache/conftool/dbconfig/20241129-040547-ladsgroup.json |
[production] |
| 04:05 |
<ladsgroup@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2229.codfw.wmnet with reason: Maintenance |
[production] |
| 04:05 |
<ladsgroup@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2229.codfw.wmnet with reason: Maintenance |
[production] |
| 04:05 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2224 (T328817)', diff saved to https://phabricator.wikimedia.org/P71417 and previous config saved to /var/cache/conftool/dbconfig/20241129-040523-ladsgroup.json |
[production] |
| 04:01 |
<tstarling@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1098915|addWiki.php tweaks]], [[gerrit:1098916|Run dumpInterwiki.php locally with no changes]], [[gerrit:1098917|Prepare id.wikivoyage.org for installation (T380726 T352113)]], [[gerrit:1099065|dumpInterwiki: read from preinstall.dblist (T352113)]], [[gerrit:1099066|addWiki: Move DB_ADMIN to core]], [[gerrit:1099064|addWiki: Add UpdateSearchIndexConf |
[production] |
| 03:50 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2224', diff saved to https://phabricator.wikimedia.org/P71416 and previous config saved to /var/cache/conftool/dbconfig/20241129-035016-ladsgroup.json |
[production] |
| 03:43 |
<raymond-ndibe@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api |
[tools] |
| 03:37 |
<raymond-ndibe@cloudcumin1001> |
START - Cookbook wmcs.toolforge.component.deploy for component builds-api |
[tools] |
| 03:37 |
<raymond-ndibe@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api |
[tools] |
| 03:35 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2224', diff saved to https://phabricator.wikimedia.org/P71415 and previous config saved to /var/cache/conftool/dbconfig/20241129-033509-ladsgroup.json |
[production] |
| 03:34 |
<raymond-ndibe@cloudcumin1001> |
START - Cookbook wmcs.toolforge.component.deploy for component builds-api |
[tools] |
| 03:34 |
<raymond-ndibe@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api |
[tools] |
| 03:34 |
<raymond-ndibe@cloudcumin1001> |
START - Cookbook wmcs.toolforge.component.deploy for component builds-api |
[tools] |
| 03:31 |
<raymond-ndibe@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api |
[toolsbeta] |
| 03:27 |
<raymond-ndibe@cloudcumin1001> |
START - Cookbook wmcs.toolforge.component.deploy for component builds-api |
[toolsbeta] |
| 03:20 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2224 (T328817)', diff saved to https://phabricator.wikimedia.org/P71414 and previous config saved to /var/cache/conftool/dbconfig/20241129-032002-ladsgroup.json |
[production] |
| 03:17 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db2224 (T328817)', diff saved to https://phabricator.wikimedia.org/P71413 and previous config saved to /var/cache/conftool/dbconfig/20241129-031705-ladsgroup.json |
[production] |
| 03:16 |
<ladsgroup@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2224.codfw.wmnet with reason: Maintenance |
[production] |
| 03:16 |
<ladsgroup@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2224.codfw.wmnet with reason: Maintenance |
[production] |
| 03:16 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2217 (T328817)', diff saved to https://phabricator.wikimedia.org/P71412 and previous config saved to /var/cache/conftool/dbconfig/20241129-031642-ladsgroup.json |
[production] |
| 03:04 |
<tstarling@deploy2002> |
scap failed: <KeyError> '1 dbs from /srv/mediawiki-staging/wikiversions.json are missing from /srv/mediawiki-staging/dblists/all.dblist: idwikivoyage' (scap version: 4.129.0) (duration: 00m 00s) |
[production] |
| 03:01 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2217', diff saved to https://phabricator.wikimedia.org/P71411 and previous config saved to /var/cache/conftool/dbconfig/20241129-030133-ladsgroup.json |
[production] |
| 02:46 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2217', diff saved to https://phabricator.wikimedia.org/P71410 and previous config saved to /var/cache/conftool/dbconfig/20241129-024625-ladsgroup.json |
[production] |
| 02:31 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2217 (T328817)', diff saved to https://phabricator.wikimedia.org/P71409 and previous config saved to /var/cache/conftool/dbconfig/20241129-023118-ladsgroup.json |
[production] |
| 02:28 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db2217 (T328817)', diff saved to https://phabricator.wikimedia.org/P71408 and previous config saved to /var/cache/conftool/dbconfig/20241129-022822-ladsgroup.json |
[production] |
| 02:28 |
<ladsgroup@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2217.codfw.wmnet with reason: Maintenance |
[production] |
| 02:28 |
<ladsgroup@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2217.codfw.wmnet with reason: Maintenance |
[production] |
| 02:27 |
<ladsgroup@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2197.codfw.wmnet with reason: Maintenance |
[production] |
| 02:26 |
<ladsgroup@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2197.codfw.wmnet with reason: Maintenance |
[production] |