|
2019-01-10
ยง
|
| 20:47 |
<marxarelli> |
both mediawiki error rates and 500 response rates have subsided back to pre-deploy levels |
[production] |
| 20:19 |
<marxarelli> |
seeing increase in "60 second timed out" error rate and rise in 503 rate, as was the case with group1 deployment. continuing to monitor |
[production] |
| 20:11 |
<dduvall@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.33.0-wmf.12 |
[production] |
| 20:11 |
<gehel> |
restart blazegraph on wdqs1009 to validate new config |
[production] |
| 20:02 |
<tgr@deploy1001> |
Synchronized php-1.33.0-wmf.12/extensions/WikimediaEvents/modules/ve-wme/campaigns.js: SWAT: [[gerrit:483485|Remove unnecessary addPlugin wrapper (T213338)]] (duration: 00m 53s) |
[production] |
| 19:50 |
<tgr@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:481987|Remove AICaptcha settings (T186244)]] (duration: 00m 52s) |
[production] |
| 19:47 |
<tgr@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:468831|Whitelist *.*.archive.org in wgCopyUploadsDomains (T207581)]] (duration: 00m 53s) |
[production] |
| 19:41 |
<tgr> |
ran mwscript namespaceDupes.php bnwikibooks --fix (238 links fixed) |
[production] |
| 19:41 |
<volans> |
installed spicerack 0.0.12-1 on cumin2001 T205884 |
[production] |
| 19:39 |
<volans> |
uploaded spicerack_0.0.12-1_amd64.deb to apt.wikimedia.org stretch-wikimedia T205884 |
[production] |
| 19:39 |
<tgr@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:458870|Note that namespaceDupes.php maintenance script run will be needed after the deployment. (T203534)]] (duration: 00m 53s) |
[production] |
| 19:14 |
<marostegui> |
Deploy schema change on dbstore1001 - T85757 |
[production] |
| 19:13 |
<marostegui> |
Deploy schema change on dbstore1002 - T85757 |
[production] |
| 18:57 |
<tzatziki> |
deleting three files for legal compliance |
[production] |
| 18:52 |
<anomie@mwmaint1002> |
Running migrateActors.php on test wikis and mediawikiwiki for T188327. This may cause lag in codfw. |
[production] |
| 18:47 |
<marostegui> |
Deploy schema change on s1 codfw master (db2048) with replication, this will generate lag on s1 codfw - T85757 |
[production] |
| 18:46 |
<marostegui> |
Stop replication on s1 codfw master for a schema change - T85757 |
[production] |
| 18:37 |
<marostegui> |
Stop replication on s8 codfw master for a schema change - T85757 |
[production] |
| 18:30 |
<marostegui> |
Upgrade mysql and kernel on db2060 |
[production] |
| 18:30 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2053, db2060 for kernel and mysql upgrade (duration: 00m 51s) |
[production] |
| 18:13 |
<marostegui> |
Stop MySQL on db2046 for kernel upgrade |
[production] |
| 18:12 |
<marostegui> |
The above change was db2053 and not db2060 |
[production] |
| 18:11 |
<marostegui> |
Stop MySQL on db2053 and db2060 for mysql and kernel upgrade |
[production] |
| 18:11 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Depool db2053, db2060 for kernel and mysql upgrade (duration: 00m 53s) |
[production] |
| 17:50 |
<jynus@deploy1001> |
Synchronized wmf-config/db-codfw.php: repool es2015 (duration: 00m 53s) |
[production] |
| 17:49 |
<marostegui> |
Deploy schema change on db2053 - T210713 |
[production] |
| 17:33 |
<marostegui> |
Deploy schema change on db2046 - T210713 |
[production] |
| 16:59 |
<jynus> |
stop and upgrade es2015 |
[production] |
| 16:52 |
<jynus@deploy1001> |
Synchronized wmf-config/db-codfw.php: depool es2015 (duration: 00m 52s) |
[production] |
| 16:41 |
<onimisionipe> |
data transfer from wdqs1004 -> wdqs1006 completed! - T213361 |
[production] |
| 16:32 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T159708 Enable Structured Data on Commons, captions-only (duration: 00m 53s) |
[production] |
| 16:17 |
<James_F> |
T180981 Placed patch to enable WBMI on Commons on mwdebug1002 |
[production] |
| 16:13 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T180981 Add Commons to wikis with WikibaseMediaInfo installed (duration: 00m 52s) |
[production] |
| 16:11 |
<jforrester@deploy1001> |
Synchronized dblists/wikidatarepo.dblist: T180981 Add Commons to wikis with WikibaseRepo installed (duration: 00m 54s) |
[production] |
| 16:04 |
<James_F> |
T180981 Placed patch to install but not enable WBMI on Commons on mwdebug1002 |
[production] |
| 15:56 |
<marostegui> |
Deploy schema change on db1068 (s4 master) - T86338 |
[production] |
| 15:31 |
<fsero> |
rollbacking last zotero codfw deployment |
[production] |
| 15:27 |
<marostegui> |
Deploy schema change on db1067 (s1 master) - T86338 T202167 |
[production] |
| 15:26 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1080 T86338 T202167 (duration: 00m 49s) |
[production] |
| 15:24 |
<addshore> |
T208330, MariaDB [testcommonswiki]> TRUNCATE TABLE wb_terms; # Was https://phabricator.wikimedia.org/P7973 |
[production] |
| 15:22 |
<fsero@deploy1001> |
scap-helm zotero upgrade production -f /srv/scap-helm/zotero/zotero-values-codfw.yaml /srv/deployment-charts/charts/zotero-0.0.1.tgz [namespace: zotero, clusters: codfw] |
[production] |
| 15:21 |
<fsero@deploy1001> |
scap-helm zotero upgrade -f /srv/scap-helm/zotero/zotero-values-codfw.yaml /srv/deployment-charts/charts/zotero-0.0.1.tgz [namespace: zotero, clusters: codfw] |
[production] |
| 15:20 |
<addshore@deploy1001> |
Synchronized php-1.33.0-wmf.12/extensions/Wikibase/repo/includes/Content: [[gerrit:483388|T208330 dont write to wb_terms for mediainfo]] (duration: 00m 54s) |
[production] |
| 15:12 |
<addshore@deploy1001> |
Synchronized php-1.33.0-wmf.9/extensions/Wikibase/repo/includes/Content: [[gerrit:483388|T208330 dont write to wb_terms for mediainfo]] (duration: 00m 55s) |
[production] |
| 14:59 |
<marostegui> |
Deploy schema change on db1080 - T86338 T202167 |
[production] |
| 14:59 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1080 T86338 T202167 (duration: 00m 52s) |
[production] |
| 14:55 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1114 T86338 T202167 (duration: 00m 52s) |
[production] |
| 14:42 |
<fsero@deploy1001> |
scap-helm zotero finished |
[production] |
| 14:42 |
<fsero@deploy1001> |
scap-helm zotero cluster staging completed |
[production] |
| 14:42 |
<fsero@deploy1001> |
scap-helm zotero upgrade staging -f /srv/scap-helm/zotero/zotero-values-staging.yaml /srv/deployment-charts/charts/zotero-0.0.1.tgz [namespace: zotero, clusters: staging] |
[production] |