|
2022-01-24
ยง
|
| 19:36 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 27c5ab3: Enable migration mode on euwiki (T299927) (duration: 00m 48s) |
[production] |
| 19:35 |
<urbanecm@deploy1002> |
Synchronized php-1.38.0-wmf.18/skins/Vector/: 4f430a8: Respect useskin when operating in MigrationMode (T299171; 2/2) (duration: 00m 48s) |
[production] |
| 19:35 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 19:35 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 19:34 |
<urbanecm@deploy1002> |
Synchronized php-1.38.0-wmf.18/skins/Vector/includes/Constants.php: 4f430a8: Respect useskin when operating in MigrationMode (T299171; 1/2) (duration: 00m 48s) |
[production] |
| 19:33 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 19:28 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 19:27 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 19:27 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 19:25 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 19:22 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: bf2981b6200d760eef8bcc70228a0207ebf9d7cb: commonswiki: Add ala-images.s3.ap-southeast-2.amazonaws.com to the wgCopyUploadsDomains allowlist (T299825) (duration: 00m 49s) |
[production] |
| 19:20 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 19:19 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 19:19 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 19:19 |
<ottomata> |
kill mysqld on an-test-coord1001 - 19:19:04 [@an-test-coord1001:/etc] $ sudo kill 42433 |
[analytics] |
| 19:18 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 2029c35b8a63e8f08b0ce5e0238c2dbee8854377: Disable RelatedArticles on ptwikinews (T299873) (duration: 00m 49s) |
[production] |
| 19:16 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 19:11 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 19:10 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 19:10 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 19:09 |
<taavi> |
deleted centralauth.global_user_groups for 10 non-existent users T299650 |
[production] |
| 19:09 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: df86dd42c18809357583f9d094af5a3d6f33c32d: Create Draft namespace for bgwiki (T299224) (duration: 00m 49s) |
[production] |
| 19:09 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 19:03 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 19:02 |
<razzi> |
razzi@an-test-coord1001:~$ sudo systemctl stop presto-server |
[analytics] |
| 19:00 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 18:23 |
<razzi> |
downtime an-coord1001 while attempting to fix /srv partition |
[analytics] |
| 18:22 |
<razzi@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on an-test-coord1001.eqiad.wmnet with reason: Unmounting /srv to try to repair the filesystem |
[production] |
| 18:22 |
<razzi@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on an-test-coord1001.eqiad.wmnet with reason: Unmounting /srv to try to repair the filesystem |
[production] |
| 17:50 |
<cmjohnson1> |
updating firmware on ganeti1013 T299527 |
[production] |
| 17:44 |
<wm-bot> |
reconfiguring the grid by using grid-configurator - cookbook ran by arturo@nostromo |
[tools] |
| 17:34 |
<wm-bot> |
<lucaswerkmeister> git -C ~/public_html/w/extensions/PluggableAuth/ checkout cdf73daf11^ # T299934 |
[tools.notwikilambda] |
| 17:28 |
<hashar> |
Nuke castor caches on integration-castor03 : sudo rm -fR /srv/jenkins-workspace/caches/castor-mw-ext-and-skins/master/{quibble-vendor-mysql-php72-selenium-docker,wmf-quibble-selenium-php72-docker} # T299933 |
[releng] |
| 17:28 |
<hashar> |
Nuke castor caches on integration-castor03 : sudo rm -fR /srv/jenkins-workspace/caches/castor-mw-ext-and-skins/master/{quibble-vendor-mysql-php72-selenium-docker,wmf-quibble-selenium-php72-docker} |
[releng] |
| 17:25 |
<wm-bot> |
<lucaswerkmeister> git -C ~/public_html/w/extensions/PluggableAuth/ checkout 8b278afaca # T299934 |
[tools.notwikilambda] |
| 17:24 |
<elukey@deploy1002> |
helmfile [staging] DONE helmfile.d/services/api-gateway: sync on staging |
[production] |
| 17:24 |
<elukey@deploy1002> |
helmfile [staging] DONE helmfile.d/services/api-gateway: sync on production |
[production] |
| 17:24 |
<elukey@deploy1002> |
helmfile [staging] START helmfile.d/services/api-gateway: sync on staging |
[production] |
| 17:24 |
<wm-bot> |
<lucaswerkmeister> kubectl delete deployment update # T299934 |
[tools.notwikilambda] |
| 17:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1101:3318 (T285149)', diff saved to https://phabricator.wikimedia.org/P19074 and previous config saved to /var/cache/conftool/dbconfig/20220124-170312-marostegui.json |
[production] |
| 16:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1101:3318', diff saved to https://phabricator.wikimedia.org/P19072 and previous config saved to /var/cache/conftool/dbconfig/20220124-164807-marostegui.json |
[production] |
| 16:48 |
<hnowlan> |
Running nodetool removenode for restbase2011-a |
[production] |
| 16:47 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 16:46 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 16:46 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 16:43 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 16:43 |
<jdrewniak@deploy1002> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:756622| Bumping portals to master (T128546)]] (duration: 00m 49s) |
[production] |
| 16:42 |
<jdrewniak@deploy1002> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:756622| Bumping portals to master (T128546)]] (duration: 00m 50s) |
[production] |
| 16:35 |
<elukey@deploy1002> |
helmfile [staging] DONE helmfile.d/services/api-gateway: sync on staging |
[production] |
| 16:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1101:3318', diff saved to https://phabricator.wikimedia.org/P19071 and previous config saved to /var/cache/conftool/dbconfig/20220124-163302-marostegui.json |
[production] |