|
2023-05-30
§
|
| 20:12 |
<inflatador> |
bking@wdqs2009 depool wdqs2009 until it catches up with lag |
[production] |
| 19:36 |
<bking@deploy1002> |
Finished deploy [wdqs/wdqs@dff41b7]: 0.3.124 (duration: 04m 02s) |
[production] |
| 19:32 |
<bking@deploy1002> |
Started deploy [wdqs/wdqs@dff41b7]: 0.3.124 |
[production] |
| 19:29 |
<bking@deploy1002> |
Finished deploy [wdqs/wdqs@dff41b7]: 0.3.124 (duration: 00m 54s) |
[production] |
| 19:29 |
<bking@deploy1002> |
Started deploy [wdqs/wdqs@dff41b7]: 0.3.124 |
[production] |
| 19:29 |
<bking@deploy1002> |
Finished deploy [wdqs/wdqs@dff41b7]: 0.3.124 (duration: 16m 36s) |
[production] |
| 19:11 |
<bking@deploy1002> |
Started deploy [wdqs/wdqs@dff41b7]: 0.3.124 |
[production] |
| 15:05 |
<bking@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/rdf-streaming-updater: apply |
[production] |
| 15:03 |
<bking@deploy1002> |
helmfile [codfw] START helmfile.d/services/rdf-streaming-updater: apply |
[production] |
| 14:08 |
<bking@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/rdf-streaming-updater: apply |
[production] |
| 14:06 |
<bking@deploy1002> |
helmfile [codfw] START helmfile.d/services/rdf-streaming-updater: apply |
[production] |
|
2023-05-22
§
|
| 20:11 |
<bking@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts wdqs[2010-2011].codfw.wmnet |
[production] |
| 20:09 |
<bking@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts wdqs[2010-2011].codfw.wmnet |
[production] |
| 16:35 |
<bking@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts wdqs2009.codfw.wmnet |
[production] |
| 16:35 |
<bking@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host wdqs2009.codfw.wmnet |
[production] |
| 15:57 |
<bking@cumin1001> |
START - Cookbook sre.hosts.reboot-single for host wdqs2009.codfw.wmnet |
[production] |
| 15:56 |
<bking@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts wdqs2009.codfw.wmnet |
[production] |
| 14:32 |
<bking@deploy1002> |
helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply |
[production] |
| 14:31 |
<bking@deploy1002> |
helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply |
[production] |
| 14:10 |
<bking@deploy1002> |
helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply |
[production] |
| 14:10 |
<bking@deploy1002> |
helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply |
[production] |
|
2023-05-18
§
|
| 22:59 |
<bking@cumin1001> |
END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.UPGRADE (3 nodes at a time) for ElasticSearch cluster search_eqiad: eqiad plugin upgrade - bking@cumin1001 - T332355 |
[production] |
| 20:36 |
<bking@cumin1001> |
START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (3 nodes at a time) for ElasticSearch cluster search_eqiad: eqiad plugin upgrade - bking@cumin1001 - T332355 |
[production] |
| 20:33 |
<bking@cumin1001> |
END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.UPGRADE (3 nodes at a time) for ElasticSearch cluster search_codfw: codfw plugin upgrade - bking@cumin1001 - T332355 |
[production] |
| 18:55 |
<bking@cumin1001> |
END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.UPGRADE (2 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic elasticsearch and plugin upgrade - bking@cumin1001 - T332355 |
[production] |
| 18:11 |
<bking@cumin1001> |
START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (3 nodes at a time) for ElasticSearch cluster search_codfw: codfw plugin upgrade - bking@cumin1001 - T332355 |
[production] |
| 18:09 |
<bking@cumin1001> |
START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (2 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic elasticsearch and plugin upgrade - bking@cumin1001 - T332355 |
[production] |
| 18:07 |
<bking@cumin1001> |
END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge elasticsearch and plugin upgrade - bking@cumin1001 - T274204 |
[production] |
| 17:59 |
<bking@cumin1001> |
START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge elasticsearch and plugin upgrade - bking@cumin1001 - T274204 |
[production] |
| 15:57 |
<inflatador> |
bking@cumin1001 starting rolling restart of wcqs for java updates T334470 |
[production] |
|
2023-05-17
§
|
| 21:44 |
<bking@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host wdqs2012.codfw.wmnet |
[production] |
| 21:26 |
<bking@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12 days, 0:00:00 on wdqs2012.codfw.wmnet with reason: attempting WDQS stack on bullseye |
[production] |
| 21:26 |
<bking@cumin1001> |
START - Cookbook sre.hosts.downtime for 12 days, 0:00:00 on wdqs2012.codfw.wmnet with reason: attempting WDQS stack on bullseye |
[production] |
| 20:59 |
<bking@cumin1001> |
START - Cookbook sre.hosts.reboot-single for host wdqs2012.codfw.wmnet |
[production] |
| 20:13 |
<bking@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts wdqs2012.codfw.wmnet |
[production] |
| 20:12 |
<bking@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts wdqs2012.codfw.wmnet |
[production] |
| 20:07 |
<bking@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts wdqs2012.codfw.wmnet |
[production] |
| 19:54 |
<bking@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts wdqs2012.codfw.wmnet |
[production] |
| 19:41 |
<inflatador> |
bking@wdqs2012 depooling to attempt firmware update T331297 |
[production] |
| 14:10 |
<bking@deploy1002> |
helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply |
[production] |
| 14:09 |
<bking@deploy1002> |
helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply |
[production] |
| 14:09 |
<bking@deploy1002> |
helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply |
[production] |
| 14:08 |
<bking@deploy1002> |
helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply |
[production] |