|
2024-10-10
ยง
|
| 21:30 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2172', diff saved to https://phabricator.wikimedia.org/P69651 and previous config saved to /var/cache/conftool/dbconfig/20241010-213029-arnaudb.json |
[production] |
| 21:15 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2172 (T367781)', diff saved to https://phabricator.wikimedia.org/P69650 and previous config saved to /var/cache/conftool/dbconfig/20241010-211522-arnaudb.json |
[production] |
| 21:05 |
<aqu@deploy2002> |
Finished deploy [airflow-dags/analytics@c9a2532]: Webrequest-Refine fix [airflow-dags@c9a2532e] (duration: 00m 51s) |
[production] |
| 21:04 |
<aqu@deploy2002> |
Started deploy [airflow-dags/analytics@c9a2532]: Webrequest-Refine fix [airflow-dags@c9a2532e] |
[production] |
| 21:04 |
<thcipriani@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1079335|Update VE core submodule to master (c98f3a542) (T376901)]] (duration: 08m 56s) |
[production] |
| 20:59 |
<thcipriani@deploy2002> |
jforrester, thcipriani: Continuing with sync |
[production] |
| 20:57 |
<thcipriani@deploy2002> |
jforrester, thcipriani: Backport for [[gerrit:1079335|Update VE core submodule to master (c98f3a542) (T376901)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 20:55 |
<thcipriani@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1079335|Update VE core submodule to master (c98f3a542) (T376901)]] |
[production] |
| 20:27 |
<eileen> |
config revision changed from 150b02a9 to 3c6d2054 |
[production] |
| 20:23 |
<thcipriani@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1076058|REST: Make experimental endpoints available on beta and testwiki (T375512)]] (duration: 08m 34s) |
[production] |
| 20:18 |
<thcipriani@deploy2002> |
bpirkle, thcipriani: Continuing with sync |
[production] |
| 20:16 |
<thcipriani@deploy2002> |
bpirkle, thcipriani: Backport for [[gerrit:1076058|REST: Make experimental endpoints available on beta and testwiki (T375512)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 20:14 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Depooling db2172 (T367781)', diff saved to https://phabricator.wikimedia.org/P69649 and previous config saved to /var/cache/conftool/dbconfig/20241010-201456-arnaudb.json |
[production] |
| 20:14 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2172.codfw.wmnet with reason: Maintenance |
[production] |
| 20:14 |
<thcipriani@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1076058|REST: Make experimental endpoints available on beta and testwiki (T375512)]] |
[production] |
| 20:14 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db2172.codfw.wmnet with reason: Maintenance |
[production] |
| 20:14 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2155 (T367781)', diff saved to https://phabricator.wikimedia.org/P69648 and previous config saved to /var/cache/conftool/dbconfig/20241010-201433-arnaudb.json |
[production] |
| 20:05 |
<eileen> |
civicrm upgraded from 07dee21c to ff3144dd |
[production] |
| 19:59 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2155', diff saved to https://phabricator.wikimedia.org/P69647 and previous config saved to /var/cache/conftool/dbconfig/20241010-195926-arnaudb.json |
[production] |
| 19:44 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2155', diff saved to https://phabricator.wikimedia.org/P69646 and previous config saved to /var/cache/conftool/dbconfig/20241010-194419-arnaudb.json |
[production] |
| 19:43 |
<aqu@deploy2002> |
Finished deploy [airflow-dags/analytics_test@4b69f50]: Stage Webrequest-Refine fix on test cluster [airflow-dags@4b69f503] (duration: 00m 13s) |
[production] |
| 19:43 |
<aqu@deploy2002> |
Started deploy [airflow-dags/analytics_test@4b69f50]: Stage Webrequest-Refine fix on test cluster [airflow-dags@4b69f503] |
[production] |
| 19:29 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2155 (T367781)', diff saved to https://phabricator.wikimedia.org/P69645 and previous config saved to /var/cache/conftool/dbconfig/20241010-192912-arnaudb.json |
[production] |
| 19:23 |
<rzl@deploy2002> |
Finished scap sync-world: chart version bump for 1078720 (duration: 02m 09s) |
[production] |
| 19:21 |
<rzl@deploy2002> |
Started scap sync-world: chart version bump for 1078720 |
[production] |
| 19:06 |
<eileen> |
config revision changed from ae4a5be9 to 150b02a9 |
[production] |
| 18:50 |
<papaul> |
maintenance on mr1-eqiad complete |
[production] |
| 18:44 |
<eileen> |
tools upgraded from 632bf430 to 62f2d170 |
[production] |
| 18:29 |
<eileen> |
tools upgraded from e9c05e30 to 632bf430 |
[production] |
| 18:28 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Depooling db2155 (T367781)', diff saved to https://phabricator.wikimedia.org/P69644 and previous config saved to /var/cache/conftool/dbconfig/20241010-182846-arnaudb.json |
[production] |
| 18:28 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2187.codfw.wmnet with reason: Maintenance |
[production] |
| 18:28 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db2187.codfw.wmnet with reason: Maintenance |
[production] |
| 18:28 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2155.codfw.wmnet with reason: Maintenance |
[production] |
| 18:28 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db2155.codfw.wmnet with reason: Maintenance |
[production] |
| 18:28 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2147 (T367781)', diff saved to https://phabricator.wikimedia.org/P69643 and previous config saved to /var/cache/conftool/dbconfig/20241010-182808-arnaudb.json |
[production] |
| 18:14 |
<jhathaway@cumin1002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest1002.eqiad.wmnet |
[production] |
| 18:13 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2147', diff saved to https://phabricator.wikimedia.org/P69642 and previous config saved to /var/cache/conftool/dbconfig/20241010-181301-arnaudb.json |
[production] |
| 18:08 |
<jhathaway@cumin1002> |
START - Cookbook sre.hosts.reboot-single for host sretest1002.eqiad.wmnet |
[production] |
| 18:00 |
<papaul> |
ongoing maintenance on mr1-eqiad |
[production] |
| 17:57 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2147', diff saved to https://phabricator.wikimedia.org/P69641 and previous config saved to /var/cache/conftool/dbconfig/20241010-175754-arnaudb.json |
[production] |
| 17:57 |
<root@cumin1002> |
END (PASS) - Cookbook sre.puppet.renew-cert (exit_code=0) for dbprov1001.eqiad.wmnet: Renew puppet certificate - root@cumin1002 |
[production] |
| 17:54 |
<root@cumin1002> |
START - Cookbook sre.puppet.renew-cert for dbprov1001.eqiad.wmnet: Renew puppet certificate - root@cumin1002 |
[production] |
| 17:47 |
<swfrench@cumin2002> |
END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) pool echostore in eqiad: Repooling echostore after migration to service mesh - T376766 |
[production] |
| 17:42 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2147 (T367781)', diff saved to https://phabricator.wikimedia.org/P69640 and previous config saved to /var/cache/conftool/dbconfig/20241010-174247-arnaudb.json |
[production] |
| 17:42 |
<swfrench@cumin2002> |
START - Cookbook sre.discovery.service-route pool echostore in eqiad: Repooling echostore after migration to service mesh - T376766 |
[production] |
| 17:39 |
<swfrench@deploy2002> |
helmfile [eqiad] DONE helmfile.d/services/echostore: apply |
[production] |
| 17:39 |
<swfrench@deploy2002> |
helmfile [eqiad] START helmfile.d/services/echostore: apply |
[production] |
| 17:38 |
<swfrench-wmf> |
removing echostore eqiad deployment (depooled) to unblock breaking change - T376766 |
[production] |
| 17:34 |
<bd808@deploy2002> |
helmfile [codfw] DONE helmfile.d/services/developer-portal: apply |
[production] |
| 17:34 |
<bd808@deploy2002> |
helmfile [codfw] START helmfile.d/services/developer-portal: apply |
[production] |