|
2019-09-04
§
|
| 08:56 |
<moritzm> |
upgrading mw1238-mw1258 to PHP 7.2.22 |
[production] |
| 08:42 |
<marostegui> |
Stop HAproxy on dbproxy1005 - T231967 |
[production] |
| 08:37 |
<moritzm> |
upgrading API canaries in eqiad to 7.2.22 |
[production] |
| 08:26 |
<marostegui> |
Reboot db1135 to pick up new kernel - T231403 |
[production] |
| 07:50 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Remove db2047 from config T231852 (duration: 00m 54s) |
[production] |
| 07:49 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Remove db2047 from config T231852 (duration: 00m 57s) |
[production] |
| 07:21 |
<mutante> |
ununpentium - a2dismod ssl - systemctl restart apache2 |
[production] |
| 05:53 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 05:51 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 02:46 |
<krinkle@deploy1001> |
Synchronized php-1.34.0-wmf.21/resources/src/startup/mediawiki.js: 8a1b13026 (duration: 00m 55s) |
[production] |
| 02:42 |
<krinkle@deploy1001> |
Synchronized php-1.34.0-wmf.21/resources/src/mediawiki.base/mediawiki.base.js: 8a1b13026 (duration: 00m 56s) |
[production] |
| 02:21 |
<chaomodus> |
extending downtime on netmon1002 and netmon2001, netbox1001, netbox2001, netboxdb1001 and netbox2001 should be stable but are still being debugged |
[production] |
| 01:02 |
<krinkle@deploy1001> |
Synchronized wmf-config/CommonSettings.php: ed5297c10 / T217830 (duration: 00m 59s) |
[production] |
| 00:02 |
<chaomodus> |
installing and setting up netbox instances T223291 |
[production] |
|
2019-09-03
§
|
| 23:57 |
<niharika29@deploy1001> |
Synchronized wmf-config/CommonSettings.php: Revert - [bugfix]Growth experiments not loading conf properly T231935 (duration: 00m 55s) |
[production] |
| 23:56 |
<niharika29@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Revert - [bugfix]Growth experiments not loading conf properly T231935 (duration: 00m 55s) |
[production] |
| 23:53 |
<niharika29@deploy1001> |
Synchronized php-1.34.0-wmf.21/extensions/GrowthExperiments/: Set correct merge strategy for help panel links T231935 (duration: 00m 55s) |
[production] |
| 23:52 |
<niharika29@deploy1001> |
Synchronized php-1.34.0-wmf.20/extensions/GrowthExperiments/: Set correct merge strategy for help panel links T231935 (duration: 00m 56s) |
[production] |
| 23:42 |
<niharika29@deploy1001> |
Synchronized php-1.34.0-wmf.20/tests/phpunit/: Allow CompositeBlock::appliesToRight to return null when unsure T229417, T231145 (duration: 00m 57s) |
[production] |
| 23:41 |
<niharika29@deploy1001> |
Synchronized php-1.34.0-wmf.20/includes/block: Allow CompositeBlock::appliesToRight to return null when unsure T229417, T231145 (duration: 00m 55s) |
[production] |
| 23:28 |
<niharika29@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable and configure ORES damaging and goodfaith on zhwiki T225562 (duration: 00m 58s) |
[production] |
| 23:10 |
<ebernhardson> |
production-search-eqiad all indices index.merge.policy.deletes_pct_allowed=20 |
[production] |
| 22:54 |
<jforrester@deploy1001> |
Synchronized wmf-config/CommonSettings.php: T208694 Set CentralNotice's wgNoticeProjects for wikimedia (duration: 00m 59s) |
[production] |
| 22:45 |
<eileen> |
process-control config revision is 100334de4a adjust silverpop schedule |
[production] |
| 19:42 |
<XioNoX> |
rollback OSPF metric change on eqiad-codfw Zayo link (1320->320) |
[production] |
| 19:20 |
<fdans@deploy1001> |
Started restart [analytics/aqs/deploy@fc1d232]: (no justification provided) |
[production] |
| 19:18 |
<fdans@deploy1001> |
Started restart [analytics/aqs/deploy@fc1d232]: (no justification provided) |
[production] |
| 19:14 |
<otto@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Switch high-traffic jobs to eventgate. Take 2 - T228705 (duration: 00m 56s) |
[production] |
| 19:12 |
<ottomata> |
switching jobqueue events to eventgate-main - T228705 |
[production] |
| 18:41 |
<urbanecm@deploy1001> |
Synchronized wmf-config/: Emergency fix: GE not loading configuration properly: newbie facing feature (duration: 00m 57s) |
[production] |
| 18:35 |
<Urbanecm> |
Livetesting on mwdebug1002 |
[production] |
| 17:45 |
<James_F> |
Pulled I9b64a2bb770 into wmf.21 production on the deploy server; no need to deploy to app-servers, CI-only fix. |
[production] |
| 17:40 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: group0 to 1.34.0-wmf.21 |
[production] |
| 16:35 |
<catrope@deploy1001> |
Synchronized php-1.34.0-wmf.21/extensions/Graph/includes/ApiGraph.php: T231894 (duration: 00m 55s) |
[production] |
| 16:01 |
<joal@deploy1001> |
Finished deploy [analytics/refinery@8b17711]: Fixes for regualr analytics deploy (duration: 136m 59s) |
[production] |
| 15:55 |
<reedy@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T227260 (duration: 00m 54s) |
[production] |
| 15:32 |
<ebernhardson> |
unban elastic1027 from production-search-eqiad |
[production] |
| 15:07 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: testwiki 1.34.0-wmf.21 for T231894 - T220746 |
[production] |
| 14:57 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: Rollback group0 to 1.34.0-wmf.21 - T220746 |
[production] |
| 14:45 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: group0 to 1.34.0-wmf.21 - T220746 |
[production] |
| 14:39 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Promote db1133 as wikitech master T229657 (duration: 00m 54s) |
[production] |
| 14:28 |
<hashar@deploy1001> |
Finished scap: testwiki to 1.34.0-wmf.21 and rebuild l10n cache - T220746 (duration: 50m 09s) |
[production] |
| 14:21 |
<moritzm> |
upgrading app server canaries to PHP 7.2.22 T230024 |
[production] |
| 13:44 |
<joal@deploy1001> |
Started deploy [analytics/refinery@8b17711]: Fixes for regualr analytics deploy |
[production] |
| 13:38 |
<hashar@deploy1001> |
Started scap: testwiki to 1.34.0-wmf.21 and rebuild l10n cache - T220746 |
[production] |
| 13:26 |
<hashar> |
Gerrit should be fine again, apparently was due to the wmf branch cut taking too much resources (sic) - T231872 filled to investigate |
[production] |
| 13:25 |
<hashar> |
1.34.0-wmf.21 cut |
[production] |
| 13:16 |
<hashar> |
Gerrit has some random times out from time to time (no reason) |
[production] |
| 13:14 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1073 from wikitech T229657', diff saved to https://phabricator.wikimedia.org/P9038 and previous config saved to /var/cache/conftool/dbconfig/20190903-131456-marostegui.json |
[production] |
| 13:13 |
<marostegui> |
Re-enable puppet on db1073 and db1133 T229657 |
[production] |