|
2022-09-26
§
|
| 07:04 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 06:49 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 06:45 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 06:45 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 06:41 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 06:36 |
<elukey> |
clean up my old home dir on matomo1002, ran `apt-get clean` + some other clean up steps on matomo1002 to free space on the root partition |
[production] |
| 06:32 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: d2d2c08fc6e0dd5c0c85fbe31f85201721871aa9: eswiki: Enable structured mentor list (T310905) (duration: 04m 30s) |
[production] |
| 06:31 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 06:30 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 06:30 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 06:29 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
|
2022-09-25
§
|
| 17:29 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1053.eqiad.wmnet with OS bullseye |
[production] |
| 17:08 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1053.eqiad.wmnet with reason: host reimage |
[production] |
| 17:05 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1053.eqiad.wmnet with reason: host reimage |
[production] |
| 16:51 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudvirt1053.eqiad.wmnet with OS bullseye |
[production] |
| 16:49 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1052.eqiad.wmnet with OS bullseye |
[production] |
| 16:23 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1052.eqiad.wmnet with reason: host reimage |
[production] |
| 16:20 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1052.eqiad.wmnet with reason: host reimage |
[production] |
| 16:06 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudvirt1052.eqiad.wmnet with OS bullseye |
[production] |
| 15:59 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1052.eqiad.wmnet with OS bullseye |
[production] |
| 15:31 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1052.eqiad.wmnet with reason: host reimage |
[production] |
| 15:26 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1052.eqiad.wmnet with reason: host reimage |
[production] |
| 15:26 |
<taavi@deploy1002> |
Finished deploy [horizon/deploy@9d02cd6]: wmf-proxy-dashboard now uses the dynamicproxy api to fetch zone data (duration: 02m 44s) |
[production] |
| 15:23 |
<taavi@deploy1002> |
Started deploy [horizon/deploy@9d02cd6]: wmf-proxy-dashboard now uses the dynamicproxy api to fetch zone data |
[production] |
| 15:22 |
<taavi@deploy1002> |
Finished deploy [horizon/deploy@9d02cd6] (dev): wmf-proxy-dashboard now uses the dynamicproxy api to fetch zone data (duration: 01m 11s) |
[production] |
| 15:20 |
<taavi@deploy1002> |
Started deploy [horizon/deploy@9d02cd6] (dev): wmf-proxy-dashboard now uses the dynamicproxy api to fetch zone data |
[production] |
| 15:15 |
<taavi@deploy1002> |
Finished deploy [horizon/deploy@9d02cd6] (dev): wmf-proxy-dashboard now uses the dynamicproxy api to fetch zone data (duration: 01m 10s) |
[production] |
| 15:14 |
<taavi@deploy1002> |
Started deploy [horizon/deploy@9d02cd6] (dev): wmf-proxy-dashboard now uses the dynamicproxy api to fetch zone data |
[production] |
| 15:13 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudvirt1052.eqiad.wmnet with OS bullseye |
[production] |
|
2022-09-23
§
|
| 19:10 |
<mforns@deploy1002> |
Finished deploy [airflow-dags/analytics@4c973d6]: (no justification provided) (duration: 00m 12s) |
[production] |
| 19:10 |
<mforns@deploy1002> |
Started deploy [airflow-dags/analytics@4c973d6]: (no justification provided) |
[production] |
| 17:49 |
<nokafor@deploy1002> |
Finished deploy [airflow-dags/analytics@7620b25]: (no justification provided) (duration: 00m 10s) |
[production] |
| 17:48 |
<nokafor@deploy1002> |
Started deploy [airflow-dags/analytics@7620b25]: (no justification provided) |
[production] |
| 13:39 |
<hashar@deploy1002> |
Finished scap: Backport for [[gerrit:834531|Stop using Elastica::Type and set the target indices (T318356)]] (duration: 07m 10s) |
[production] |
| 13:37 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 13:36 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 13:36 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 13:35 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 13:32 |
<hashar@deploy1002> |
hashar and hashar: Backport for [[gerrit:834531|Stop using Elastica::Type and set the target indices (T318356)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet |
[production] |
| 13:31 |
<hashar@deploy1002> |
Started scap: Backport for [[gerrit:834531|Stop using Elastica::Type and set the target indices (T318356)]] |
[production] |
| 13:29 |
<taavi@deploy1002> |
Finished deploy [horizon/deploy@9d02cd6]: wmf-proxy-dashboard improved error handling (duration: 03m 06s) |
[production] |
| 13:26 |
<taavi@deploy1002> |
Started deploy [horizon/deploy@9d02cd6]: wmf-proxy-dashboard improved error handling |
[production] |
| 13:24 |
<taavi@deploy1002> |
Finished deploy [horizon/deploy@9d02cd6] (dev): wmf-proxy-dashboard improved error handling (duration: 01m 11s) |
[production] |
| 13:23 |
<taavi@deploy1002> |
Started deploy [horizon/deploy@9d02cd6] (dev): wmf-proxy-dashboard improved error handling |
[production] |
| 09:26 |
<jynus> |
stopping db1117:s3 for maintenance T315713 |
[production] |
| 08:51 |
<Emperor> |
rebalance ms-eqiad swift rings T294550 |
[production] |
| 07:36 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db[2134,2160].codfw.wmnet,db[1117,1159].eqiad.wmnet with reason: Grants fixing |
[production] |
| 07:36 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db[2134,2160].codfw.wmnet,db[1117,1159].eqiad.wmnet with reason: Grants fixing |
[production] |
| 06:10 |
<marostegui> |
Shutdown db1189 T317662 |
[production] |
| 06:09 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on db1189.eqiad.wmnet with reason: on site maintenance |
[production] |