|
2022-10-31
§
|
| 07:30 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2124', diff saved to https://phabricator.wikimedia.org/P37051 and previous config saved to /var/cache/conftool/dbconfig/20221031-073042-marostegui.json |
[production] |
| 07:29 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |
| 07:15 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2124', diff saved to https://phabricator.wikimedia.org/P37050 and previous config saved to /var/cache/conftool/dbconfig/20221031-071536-marostegui.json |
[production] |
| 07:14 |
<kartik@deploy1002> |
Finished scap: Backport for [[gerrit:845573|Enable Section Translation in Hawaiian, Pashto and Xhosa WPs (T317289)]] (duration: 06m 48s) |
[production] |
| 07:14 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 07:13 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mw-debug: apply |
[production] |
| 07:13 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 07:12 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |
| 07:08 |
<kartik@deploy1002> |
kartik and kartik: Backport for [[gerrit:845573|Enable Section Translation in Hawaiian, Pashto and Xhosa WPs (T317289)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet |
[production] |
| 07:07 |
<kartik@deploy1002> |
Started scap: Backport for [[gerrit:845573|Enable Section Translation in Hawaiian, Pashto and Xhosa WPs (T317289)]] |
[production] |
| 07:02 |
<ryankemper> |
[WDQS] `ryankemper@wdqs1007:~$ sudo systemctl restart wdqs-blazegraph.service` |
[production] |
| 07:00 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2124 (T321123)', diff saved to https://phabricator.wikimedia.org/P37049 and previous config saved to /var/cache/conftool/dbconfig/20221031-070029-marostegui.json |
[production] |
| 06:58 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2124 (T321123)', diff saved to https://phabricator.wikimedia.org/P37048 and previous config saved to /var/cache/conftool/dbconfig/20221031-065817-marostegui.json |
[production] |
| 06:58 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2124.codfw.wmnet with reason: Maintenance |
[production] |
| 06:58 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db2124.codfw.wmnet with reason: Maintenance |
[production] |
| 06:57 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2117 (T321123)', diff saved to https://phabricator.wikimedia.org/P37047 and previous config saved to /var/cache/conftool/dbconfig/20221031-065756-marostegui.json |
[production] |
| 06:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2117', diff saved to https://phabricator.wikimedia.org/P37046 and previous config saved to /var/cache/conftool/dbconfig/20221031-064249-marostegui.json |
[production] |
| 06:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2117 (T321123)', diff saved to https://phabricator.wikimedia.org/P37044 and previous config saved to /var/cache/conftool/dbconfig/20221031-061236-marostegui.json |
[production] |
| 06:10 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2117 (T321123)', diff saved to https://phabricator.wikimedia.org/P37043 and previous config saved to /var/cache/conftool/dbconfig/20221031-061026-marostegui.json |
[production] |
| 06:10 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2117.codfw.wmnet with reason: Maintenance |
[production] |
| 06:09 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db2117.codfw.wmnet with reason: Maintenance |
[production] |
| 06:08 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1122.eqiad.wmnet with reason: Maintenance |
[production] |
| 06:08 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1122.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:42 |
<kartik@deploy1002> |
helmfile [staging] DONE helmfile.d/services/cxserver: apply |
[production] |
| 05:41 |
<kartik@deploy1002> |
helmfile [staging] START helmfile.d/services/cxserver: apply |
[production] |
| 03:10 |
<wm-bot> |
<anticomposite> ./stewardbots/StewardBot/manage.sh restart # Ping timeout |
[tools.stewardbots] |
| 00:12 |
<Deus> |
Removing role::simplelamp2 from ceres-01 |
[wikisp] |
|
2022-10-30
§
|
| 11:00 |
<wm-bot> |
<lucaswerkmeister> lowered notwikilambda quotas: count/deployments.apps from 5 to 3, services from 4 to 1 (T321990) |
[tools.notwikilambda] |
| 10:58 |
<wm-bot> |
<lucaswerkmeister> kubectl delete service function-evaluator function-orchestrator pygments-server # T314880 |
[tools.notwikilambda] |
| 10:34 |
<wm-bot> |
<lucaswerkmeister> kubectl delete deployment function-evaluator function-orchestrator pygments-server (T314880) |
[tools.notwikilambda] |
| 10:34 |
<wm-bot> |
<lucaswerkmeister> kubectl delete deployment function-evaluator function-orchestrator pygments-server |
[tools.notwikilambda] |
| 10:30 |
<wm-bot> |
<lucaswerkmeister> Double IRC messages to other bridges |
[tools.bridgebot] |
| 10:22 |
<wm-bot> |
<lucaswerkmeister> start webservice again (PHP 7.4 now) with new lighttpd config to rewrite everything to a very bare index.html explaining the shutdown (T314880); partial SQL dump available in public_html too |
[tools.notwikilambda] |
| 10:04 |
<wm-bot> |
<lucaswerkmeister> stopped webservice, wiki is being retired (T314880) |
[tools.notwikilambda] |
| 10:04 |
<Deus> |
Rebuild mars-01 (T321763) |
[wikisp] |
|
2022-10-29
§
|
| 20:07 |
<wm-bot> |
<lucaswerkmeister> Double IRC messages to other bridges |
[tools.bridgebot] |
| 19:44 |
<wm-bot> |
<lucaswerkmeister> deployed 19f1300fa1 (mypy) |
[tools.speedpatrolling] |
| 19:07 |
<wm-bot> |
<lucaswerkmeister> deployed f4c38cd04e (rm dead code) |
[tools.speedpatrolling] |
| 17:30 |
<wm-bot> |
<lucaswerkmeister> deployed 4c9e42dbc8 (flake8) |
[tools.speedpatrolling] |
| 17:22 |
<wm-bot> |
<lucaswerkmeister> Double IRC messages to other bridges |
[tools.bridgebot] |
| 15:47 |
<wm-bot> |
<lucaswerkmeister> deployed ae8fce43f8 (gitignore update – pulled without webservice restart) |
[tools.speedpatrolling] |
| 15:31 |
<wm-bot> |
<anticomposite> ./SULWatcher/manage.sh restart # SULWatchers disconnected |
[tools.stewardbots] |
| 15:10 |
<wm-bot> |
<lucaswerkmeister> deployed ee9bd42513 (README update – pulled without webservice restart) |
[tools.speedpatrolling] |
| 15:08 |
<wm-bot> |
<lucaswerkmeister> deployed 9058cfad77 (use pip-tools, update dependencies) |
[tools.speedpatrolling] |
| 11:25 |
<taavi> |
deploy patch for T321971 |
[production] |
| 11:22 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 11:21 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mw-debug: apply |
[production] |
| 11:21 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 11:21 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |