|
2019-01-23
§
|
| 09:38 |
<akosiaris@deploy1001> |
scap-helm mathoid cluster codfw completed |
[production] |
| 09:38 |
<akosiaris@deploy1001> |
scap-helm mathoid cluster eqiad completed |
[production] |
| 09:38 |
<akosiaris@deploy1001> |
scap-helm mathoid upgrade -f mathoid-values.yaml production stable/mathoid [namespace: mathoid, clusters: eqiad,codfw] |
[production] |
| 09:30 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=prometheus1003.eqiad.wmnet |
[production] |
| 09:23 |
<akosiaris@deploy1001> |
scap-helm mathoid finished |
[production] |
| 09:23 |
<akosiaris@deploy1001> |
scap-helm mathoid cluster staging completed |
[production] |
| 09:23 |
<akosiaris@deploy1001> |
scap-helm mathoid upgrade -f mathoid-values.yaml --set resources.replicas=1 staging stable/mathoid [namespace: mathoid, clusters: staging] |
[production] |
| 09:22 |
<akosiaris@deploy1001> |
scap-helm mathoid finished |
[production] |
| 09:22 |
<akosiaris@deploy1001> |
scap-helm mathoid cluster staging completed |
[production] |
| 09:22 |
<akosiaris@deploy1001> |
scap-helm mathoid upgrade -f mathoid-values.yaml staging stable/mathoid [namespace: mathoid, clusters: staging] |
[production] |
| 08:55 |
<marostegui> |
Deploy schema change on s5 codfw master with replication, lag will be generated - T210713 |
[production] |
| 08:44 |
<addshore> |
addshore@mwmaint1002:~$ mwscript extensions/Cognate/maintenance/populateCognatePages.php --wiki yuewiktionary --batch-size 1000 // T214400 |
[production] |
| 08:28 |
<marostegui> |
Deploy schema change on db1061 (s6 primary master) - T210713 |
[production] |
| 08:27 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1088 T210713 (duration: 00m 55s) |
[production] |
| 08:19 |
<marostegui> |
Add dbstore1004:3314 to tendril - T210478 |
[production] |
| 08:18 |
<marostegui> |
Add dbstore1004:3314 to zarcillo - T210478 |
[production] |
| 08:12 |
<marostegui> |
Deploy schema change on db1088 T210713 |
[production] |
| 08:12 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1088 T210713 (duration: 00m 52s) |
[production] |
| 08:09 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1093 T210713 (duration: 00m 52s) |
[production] |
| 07:51 |
<marostegui> |
Compress tables on dbstore1004:3314 - T210478 |
[production] |
| 07:48 |
<marostegui> |
Deploy schema change on db1093 - T210713 |
[production] |
| 07:48 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1093 T210713 (duration: 00m 54s) |
[production] |
| 07:39 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1085 T210713 (duration: 00m 52s) |
[production] |
| 07:13 |
<marostegui> |
Deploy schema change on db1085, this will generate lag on s6 labs - T210713 |
[production] |
| 07:12 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1085 T210713 (duration: 00m 53s) |
[production] |
| 07:07 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1113:3316 T210713 (duration: 00m 52s) |
[production] |
| 06:53 |
<marostegui> |
Deploy schema change on db1113:3316 - T210713 |
[production] |
| 06:53 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1113:3316 T210713 (duration: 00m 53s) |
[production] |
| 06:25 |
<marostegui> |
Stop s4 on db1102 to clone dbstore1004 - T210478 |
[production] |
| 06:16 |
<marostegui@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Increase parsercache TTL keys from 22 to 24 days T210992 (duration: 01m 06s) |
[production] |
| 04:05 |
<tstarling@deploy1001> |
Finished scap: gerrit 480419 (duration: 19m 33s) |
[production] |
| 03:45 |
<tstarling@deploy1001> |
Started scap: gerrit 480419 |
[production] |
| 03:44 |
<tstarling@deploy1001> |
Synchronized wmf-config/PhpAutoPrepend.php: gerrit 480419 (duration: 00m 52s) |
[production] |
| 03:41 |
<tstarling@deploy1001> |
Synchronized wmf-config/profiler.php: gerrit 480419 (duration: 00m 54s) |
[production] |
| 03:40 |
<tstarling@deploy1001> |
Synchronized wmf-config/CommonSettings.php: gerrit 480419 (duration: 00m 54s) |
[production] |
| 03:38 |
<tstarling@deploy1001> |
scap failed: average error rate on 9/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/db09a36be5ed3e81155041f7d46ad040 for details) |
[production] |
| 03:36 |
<tstarling@deploy1001> |
Synchronized wmf-config/arclamp.php: gerrit 480419 (duration: 00m 54s) |
[production] |
| 03:32 |
<tstarling@deploy1001> |
Synchronized php-1.33.0-wmf.13/LocalSettings.php: gerrit 480419 (duration: 00m 54s) |
[production] |
| 03:29 |
<tstarling@deploy1001> |
Synchronized php-1.33.0-wmf.14/LocalSettings.php: gerrit 480419 (duration: 00m 52s) |
[production] |
| 03:27 |
<tstarling@deploy1001> |
Synchronized src/XWikimediaDebug.php: gerrit 480419 (duration: 00m 55s) |
[production] |
| 03:22 |
<TimStarling> |
manually edited LocalSettings.php in php-1.33.0-wmf.13 and php-1.33.0-wmf.14 to use a relative path, like in https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/480695/ |
[production] |
| 03:09 |
<tstarling@deploy1001> |
Scap failed!: Call to mwscript eval.php returned: None |
[production] |
| 01:15 |
<mutante> |
scandium - puppet run now without errors for the first time for the parsoid testing role on stretch instead of jessie. nodejs 10. - @subbu @arlolra you can start using it to replace ruthenium (T201366) |
[production] |
| 01:12 |
<mutante> |
scandium - git cloning parsoid from gerrit - mediawiki/services/parsoid/deploy to /srv/deployment/parsoid/deploy ; still needs https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/484602/ (T201366) |
[production] |
| 01:05 |
<mutante> |
scandium - deleting /etc/apt/preferences.d/stretch_backports.pref ; apt-get remove nodejs ; apt-get install -t stretch-backports npm ; now has nodejs 10 and npm from backports installed (T201366) |
[production] |
| 00:58 |
<mutante> |
scandium - deleting /etc/apt/preferences.d/stretch_backports.pref ; apt-get remove nodejs |
[production] |
| 00:52 |
<ebernhardson@deploy1001> |
Synchronized php-1.33.0-wmf.13/extensions/ContentTranslation/scripts/purge-unpublished-drafts.php: SWAT T203059 ContentTranslation: Remove waitForReplication for dry-run (duration: 00m 55s) |
[production] |
| 00:40 |
<ebernhardson@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT T213851 Cirrus: Setup archive index shard/replica counts (duration: 00m 54s) |
[production] |
| 00:05 |
<gtirloni> |
T209527 disabled notifications for cloudstore100{8,9} |
[production] |