|
2024-07-16
ยง
|
| 14:29 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1212 (T367781)', diff saved to https://phabricator.wikimedia.org/P66633 and previous config saved to /var/cache/conftool/dbconfig/20240716-142953-arnaudb.json |
[production] |
| 14:25 |
<urbanecm@deploy1002> |
Started scap sync-world: Backport for [[gerrit:1054572|Introduce Vanish Request Flow (T367329 T367726 T367728 T367729 T367744 T368177 T368285 T368368 T368372 T368611 T369489)]], [[gerrit:1054573|Pass wiki id to actor store for cross-db hasPublicLogs query (T370059)]], [[gerrit:1054574|Properly set automatic vanish performer on GlobalRenameUser (T368177)]], [[gerrit:1053373|Enable account vanishing |
[production] |
| 14:23 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Depooling db1212 (T367781)', diff saved to https://phabricator.wikimedia.org/P66632 and previous config saved to /var/cache/conftool/dbconfig/20240716-142321-arnaudb.json |
[production] |
| 14:23 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:22 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 8:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:22 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1212.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:22 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db1212.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:20 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1198 (T367781)', diff saved to https://phabricator.wikimedia.org/P66631 and previous config saved to /var/cache/conftool/dbconfig/20240716-142029-arnaudb.json |
[production] |
| 14:12 |
<jiji@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mw-api-int: apply |
[production] |
| 14:11 |
<jiji@deploy1002> |
helmfile [codfw] START helmfile.d/services/mw-api-int: apply |
[production] |
| 14:10 |
<jiji@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mw-api-int: apply |
[production] |
| 14:08 |
<jiji@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mw-api-int: apply |
[production] |
| 14:07 |
<jiji@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 14:07 |
<jiji@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |
| 14:05 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1198', diff saved to https://phabricator.wikimedia.org/P66630 and previous config saved to /var/cache/conftool/dbconfig/20240716-140522-arnaudb.json |
[production] |
| 14:03 |
<cgoubert@cumin1002> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts mw2432.codfw.wmnet |
[production] |
| 13:53 |
<cgoubert@cumin1002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts mw2432.codfw.wmnet |
[production] |
| 13:50 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1198', diff saved to https://phabricator.wikimedia.org/P66629 and previous config saved to /var/cache/conftool/dbconfig/20240716-135015-arnaudb.json |
[production] |
| 13:40 |
<tgr|away> |
UTC afternoon deploys done |
[production] |
| 13:39 |
<tgr@deploy1002> |
Finished scap: Backport for [[gerrit:1036245|Handle sso.wikimedia.org domain (T365162)]] (duration: 19m 07s) |
[production] |
| 13:35 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1198 (T367781)', diff saved to https://phabricator.wikimedia.org/P66628 and previous config saved to /var/cache/conftool/dbconfig/20240716-133508-arnaudb.json |
[production] |
| 13:34 |
<tgr@deploy1002> |
tgr: Continuing with sync |
[production] |
| 13:29 |
<mforns@deploy1002> |
Finished deploy [airflow-dags/analytics@1ee55b8]: (no justification provided) (duration: 00m 30s) |
[production] |
| 13:29 |
<mforns@deploy1002> |
Started deploy [airflow-dags/analytics@1ee55b8]: (no justification provided) |
[production] |
| 13:29 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Depooling db1198 (T367781)', diff saved to https://phabricator.wikimedia.org/P66627 and previous config saved to /var/cache/conftool/dbconfig/20240716-132915-arnaudb.json |
[production] |
| 13:29 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1198.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:28 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db1198.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:28 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1189 (T367781)', diff saved to https://phabricator.wikimedia.org/P66626 and previous config saved to /var/cache/conftool/dbconfig/20240716-132853-arnaudb.json |
[production] |
| 13:22 |
<tgr@deploy1002> |
tgr: Backport for [[gerrit:1036245|Handle sso.wikimedia.org domain (T365162)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 13:20 |
<tgr@deploy1002> |
Started scap sync-world: Backport for [[gerrit:1036245|Handle sso.wikimedia.org domain (T365162)]] |
[production] |
| 13:15 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 Finished scap: Backport for [[gerrit:1052762|EventStreamConfig: Enable hive ingestion for mediawiki.page-delete (T367134)]] (duration: 10m 15s) |
[production] |
| 13:13 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1189', diff saved to https://phabricator.wikimedia.org/P66625 and previous config saved to /var/cache/conftool/dbconfig/20240716-131346-arnaudb.json |
[production] |
| 13:10 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 tchin, lucaswerkmeister-wmde: Continuing with sync |
[production] |
| 13:09 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 tchin, lucaswerkmeister-wmde: Backport for [[gerrit:1052762|EventStreamConfig: Enable hive ingestion for mediawiki.page-delete (T367134)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 13:05 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 Started scap sync-world: Backport for [[gerrit:1052762|EventStreamConfig: Enable hive ingestion for mediawiki.page-delete (T367134)]] |
[production] |
| 12:58 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1189', diff saved to https://phabricator.wikimedia.org/P66624 and previous config saved to /var/cache/conftool/dbconfig/20240716-125839-arnaudb.json |
[production] |
| 12:46 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db2130 (T367856)', diff saved to https://phabricator.wikimedia.org/P66623 and previous config saved to /var/cache/conftool/dbconfig/20240716-124604-marostegui.json |
[production] |
| 12:45 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 6:00:00 on db2130.codfw.wmnet with reason: Maintenance |
[production] |
| 12:45 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 6:00:00 on db2130.codfw.wmnet with reason: Maintenance |
[production] |
| 12:45 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2116 (T367856)', diff saved to https://phabricator.wikimedia.org/P66622 and previous config saved to /var/cache/conftool/dbconfig/20240716-124543-marostegui.json |
[production] |
| 12:43 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1189 (T367781)', diff saved to https://phabricator.wikimedia.org/P66621 and previous config saved to /var/cache/conftool/dbconfig/20240716-124332-arnaudb.json |
[production] |
| 12:30 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2116', diff saved to https://phabricator.wikimedia.org/P66620 and previous config saved to /var/cache/conftool/dbconfig/20240716-123035-marostegui.json |
[production] |
| 12:20 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1174 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P66619 and previous config saved to /var/cache/conftool/dbconfig/20240716-122039-root.json |
[production] |
| 12:15 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2116', diff saved to https://phabricator.wikimedia.org/P66618 and previous config saved to /var/cache/conftool/dbconfig/20240716-121528-marostegui.json |
[production] |
| 12:10 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) netbox to netbox-dev2003.codfw.wmnet with reason: Release v4.0.7 to netbox-next - ayounsi@cumin1002 - T336275 |
[production] |
| 12:09 |
<ayounsi@cumin1002> |
START - Cookbook sre.deploy.python-code netbox to netbox-dev2003.codfw.wmnet with reason: Release v4.0.7 to netbox-next - ayounsi@cumin1002 - T336275 |
[production] |
| 12:05 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1174 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P66617 and previous config saved to /var/cache/conftool/dbconfig/20240716-120534-root.json |
[production] |
| 12:00 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2116 (T367856)', diff saved to https://phabricator.wikimedia.org/P66616 and previous config saved to /var/cache/conftool/dbconfig/20240716-120021-marostegui.json |
[production] |
| 12:00 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db2116 (T367856)', diff saved to https://phabricator.wikimedia.org/P66615 and previous config saved to /var/cache/conftool/dbconfig/20240716-120012-marostegui.json |
[production] |
| 12:00 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 6:00:00 on db2116.codfw.wmnet with reason: Maintenance |
[production] |