|
2021-09-21
§
|
| 04:58 |
<kartik@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'cxserver' for release 'staging' . |
[production] |
| 02:35 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 02:33 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 02:05 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 02:03 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 00:16 |
<tgr> |
Evening deploys done |
[production] |
| 00:16 |
<tgr@deploy1002> |
Synchronized php-1.37.0-wmf.23/extensions/GrowthExperiments/modules/ext.growthExperiments.StructuredTask/addlink/AddLinkArticleTarget.js: Backport: [[gerrit:722449|AddLink: Skip over headings in phrase matching (T291361)]] (duration: 00m 57s) |
[production] |
| 00:04 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 00:03 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
|
2021-09-20
§
|
| 23:31 |
<ejegg> |
updated fundraising CiviCRM from e6bf81d99c to 14658445a2 |
[production] |
| 23:29 |
<bd808@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'toolhub' for release 'main' . |
[production] |
| 23:22 |
<mutante> |
LDAP - added georginaburnett-wmde to NDA group (T291391, T273780) |
[production] |
| 23:15 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 23:13 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 23:08 |
<bstorm> |
ran `echo check > /sys/block/md0/md/sync_action` on cloudcontrol1004 to check raid |
[admin] |
| 22:48 |
<andrewbogott> |
stopped puppet & mariadb on cloudcontrol1004; it was flapping |
[admin] |
| 22:44 |
<andrewbogott> |
sudo touch /tmp/galera.disabled on cloudcontrol1004, the service seems troubled there |
[admin] |
| 22:36 |
<bstorm> |
created cloudstore-nfs-01 with a floating ip and a 10GB cinder volume T291406 |
[cloudstore] |
| 22:21 |
<bd808@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'toolhub' for release 'main' . |
[production] |
| 22:14 |
<mutante> |
wdqs1004 - depool |
[production] |
| 22:10 |
<mutante> |
wdqs1004 - service wdqs-updater restart |
[production] |
| 22:06 |
<mutante> |
wdqs1004 - HTTP/1.1 503 Service Unavailable - systemctl restart wdqs-blazegraph |
[production] |
| 22:05 |
<foks> |
changing user email for MIskander (WMF)@collabwiki |
[production] |
| 21:57 |
<andrewbogott> |
moving cloudvirt1043 into the 'nfs' aggregate for T291405 |
[admin] |
| 21:41 |
<mutante> |
ms-fe1005 - systemctl start swift_dispersion_stats.service (gerrit:719285) |
[production] |
| 21:30 |
<bd808@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'toolhub' for release 'main' . |
[production] |
| 21:23 |
<wm-bot> |
<lucaswerkmeister> successfully rolled out node12 for function-orchestrator too |
[tools.notwikilambda] |
| 21:19 |
<wm-bot> |
<lucaswerkmeister> successfully rolled out node12 for function-evaluator after changing `npm ci` to `npm install` (ecfa5ab9ab) |
[tools.notwikilambda] |
| 21:00 |
<wm-bot> |
<lucaswerkmeister> tried to deploy d028cb3667 (switch to node12) for function-evaluator but the new pod never came up so I undid the rollout; will consider further steps |
[tools.notwikilambda] |
| 20:50 |
<wm-bot> |
<lucaswerkmeister> set cronjob successfulJobsHistoryLimit to 0 (0aa054b391) |
[tools.notwikilambda] |
| 20:40 |
<wm-bot> |
<lucaswerkmeister> `git -C ~/www/js/function-orchestrator/ config pull.rebase true` ditto |
[tools.notwikilambda] |
| 20:40 |
<wm-bot> |
<lucaswerkmeister> `git -C ~/www/js/function-evaluator/ config pull.rebase true` to silence `git pull` warnings (presumably due to newer git in node12 container) |
[tools.notwikilambda] |
| 19:49 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:47 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:45 |
<legoktm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Revert "Disable jQuery Migrate on group1" (T291410) (duration: 00m 56s) |
[production] |
| 17:26 |
<wm-bot> |
<root> Added missing "title" in $HOME/public_html/static/toolinfo.json. |
[tools.gerrit-patch-uploader] |
| 17:24 |
<wm-bot> |
<root> Added missing "title" in $HOME/public_html/toolinfo.json. |
[tools.gerrit-reviewer-bot] |
| 17:23 |
<wm-bot> |
<root> Fixed JSON syntax errors in $HOME/public_html/toolinfo.json. Updated URLs in same file to new toolforge.org standard. |
[tools.tsreports] |
| 17:22 |
<wm-bot> |
<root> Updated $HOME/public_html/toolinfo.json to add missing "title" and update URL to toolforge.org |
[tools.wikilinkbot] |
| 17:20 |
<wm-bot> |
<root> Fixed JSON syntax errors in $HOME/public_html/toolinfo.json. Updated URLs in same file to new toolforge.org standard. |
[tools.apersonbot] |
| 17:13 |
<bd808> |
Freed up disk space on toolhub-demo01 with `docker image prune -a` to remove all unused containers pulled to the instance (mostly old snapshots of the toolhub image). |
[toolhub] |
| 17:05 |
<bd808> |
Updated toolhub-demo.wmcloud.org to 8c8c4fb |
[toolhub] |
| 17:02 |
<legoktm> |
repooled codfw (traffic/caches) 1 week after DC switchover |
[production] |
| 16:41 |
<effie> |
upgrading php on wtp[1025-1029] to 7.2.34-18+0~20210223.60+debian10~1.gbpb21322+wmf2 - T291052 |
[production] |
| 16:28 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 16:26 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 14:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 100%: After migrating wikitech to codfw', diff saved to https://phabricator.wikimedia.org/P17305 and previous config saved to /var/cache/conftool/dbconfig/20210920-144844-root.json |
[production] |
| 14:42 |
<jiji@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'tegola-vector-tiles' for release 'main' . |
[production] |
| 14:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 75%: After migrating wikitech to codfw', diff saved to https://phabricator.wikimedia.org/P17304 and previous config saved to /var/cache/conftool/dbconfig/20210920-143340-root.json |
[production] |
| 14:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 50%: After migrating wikitech to codfw', diff saved to https://phabricator.wikimedia.org/P17303 and previous config saved to /var/cache/conftool/dbconfig/20210920-141836-root.json |
[production] |