|
2019-01-23
§
|
| 08:28 |
<marostegui> |
Deploy schema change on db1061 (s6 primary master) - T210713 |
[production] |
| 08:27 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1088 T210713 (duration: 00m 55s) |
[production] |
| 08:19 |
<marostegui> |
Add dbstore1004:3314 to tendril - T210478 |
[production] |
| 08:18 |
<marostegui> |
Add dbstore1004:3314 to zarcillo - T210478 |
[production] |
| 08:12 |
<marostegui> |
Deploy schema change on db1088 T210713 |
[production] |
| 08:12 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1088 T210713 (duration: 00m 52s) |
[production] |
| 08:09 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1093 T210713 (duration: 00m 52s) |
[production] |
| 07:51 |
<marostegui> |
Compress tables on dbstore1004:3314 - T210478 |
[production] |
| 07:48 |
<marostegui> |
Deploy schema change on db1093 - T210713 |
[production] |
| 07:48 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1093 T210713 (duration: 00m 54s) |
[production] |
| 07:39 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1085 T210713 (duration: 00m 52s) |
[production] |
| 07:13 |
<marostegui> |
Deploy schema change on db1085, this will generate lag on s6 labs - T210713 |
[production] |
| 07:12 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1085 T210713 (duration: 00m 53s) |
[production] |
| 07:07 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1113:3316 T210713 (duration: 00m 52s) |
[production] |
| 06:53 |
<marostegui> |
Deploy schema change on db1113:3316 - T210713 |
[production] |
| 06:53 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1113:3316 T210713 (duration: 00m 53s) |
[production] |
| 06:25 |
<marostegui> |
Stop s4 on db1102 to clone dbstore1004 - T210478 |
[production] |
| 06:16 |
<marostegui@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Increase parsercache TTL keys from 22 to 24 days T210992 (duration: 01m 06s) |
[production] |
| 04:05 |
<tstarling@deploy1001> |
Finished scap: gerrit 480419 (duration: 19m 33s) |
[production] |
| 03:45 |
<tstarling@deploy1001> |
Started scap: gerrit 480419 |
[production] |
| 03:44 |
<tstarling@deploy1001> |
Synchronized wmf-config/PhpAutoPrepend.php: gerrit 480419 (duration: 00m 52s) |
[production] |
| 03:41 |
<tstarling@deploy1001> |
Synchronized wmf-config/profiler.php: gerrit 480419 (duration: 00m 54s) |
[production] |
| 03:40 |
<tstarling@deploy1001> |
Synchronized wmf-config/CommonSettings.php: gerrit 480419 (duration: 00m 54s) |
[production] |
| 03:38 |
<tstarling@deploy1001> |
scap failed: average error rate on 9/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/db09a36be5ed3e81155041f7d46ad040 for details) |
[production] |
| 03:36 |
<tstarling@deploy1001> |
Synchronized wmf-config/arclamp.php: gerrit 480419 (duration: 00m 54s) |
[production] |
| 03:32 |
<tstarling@deploy1001> |
Synchronized php-1.33.0-wmf.13/LocalSettings.php: gerrit 480419 (duration: 00m 54s) |
[production] |
| 03:29 |
<tstarling@deploy1001> |
Synchronized php-1.33.0-wmf.14/LocalSettings.php: gerrit 480419 (duration: 00m 52s) |
[production] |
| 03:27 |
<tstarling@deploy1001> |
Synchronized src/XWikimediaDebug.php: gerrit 480419 (duration: 00m 55s) |
[production] |
| 03:22 |
<TimStarling> |
manually edited LocalSettings.php in php-1.33.0-wmf.13 and php-1.33.0-wmf.14 to use a relative path, like in https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/480695/ |
[production] |
| 03:09 |
<tstarling@deploy1001> |
Scap failed!: Call to mwscript eval.php returned: None |
[production] |
| 01:15 |
<mutante> |
scandium - puppet run now without errors for the first time for the parsoid testing role on stretch instead of jessie. nodejs 10. - @subbu @arlolra you can start using it to replace ruthenium (T201366) |
[production] |
| 01:12 |
<mutante> |
scandium - git cloning parsoid from gerrit - mediawiki/services/parsoid/deploy to /srv/deployment/parsoid/deploy ; still needs https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/484602/ (T201366) |
[production] |
| 01:05 |
<mutante> |
scandium - deleting /etc/apt/preferences.d/stretch_backports.pref ; apt-get remove nodejs ; apt-get install -t stretch-backports npm ; now has nodejs 10 and npm from backports installed (T201366) |
[production] |
| 00:58 |
<mutante> |
scandium - deleting /etc/apt/preferences.d/stretch_backports.pref ; apt-get remove nodejs |
[production] |
| 00:52 |
<ebernhardson@deploy1001> |
Synchronized php-1.33.0-wmf.13/extensions/ContentTranslation/scripts/purge-unpublished-drafts.php: SWAT T203059 ContentTranslation: Remove waitForReplication for dry-run (duration: 00m 55s) |
[production] |
| 00:40 |
<ebernhardson@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT T213851 Cirrus: Setup archive index shard/replica counts (duration: 00m 54s) |
[production] |
| 00:05 |
<gtirloni> |
T209527 disabled notifications for cloudstore100{8,9} |
[production] |
|
2019-01-22
§
|
| 23:09 |
<cstone> |
Updated payments-wiki from 7d4cd165d9 to ca7c280f3e |
[production] |
| 22:22 |
<twentyafterfour@deploy1001> |
Finished scap: testwikis wikis to 1.33.0-wmf.14 refs T206668 (duration: 43m 00s) |
[production] |
| 21:39 |
<twentyafterfour@deploy1001> |
Started scap: testwikis wikis to 1.33.0-wmf.14 refs T206668 |
[production] |
| 21:31 |
<twentyafterfour@deploy1001> |
Synchronized wmf-config/CommonSettings.php: deploy I91e9028d70d5a6f109f341bad553816e617b2416 (duration: 01m 39s) |
[production] |
| 20:26 |
<gehel> |
resetting cassandra authentication on maps / eqiad |
[production] |
| 20:25 |
<milimetric@deploy1001> |
Finished deploy [analytics/refinery@d806b62]: Update jar versions on modified jobs (duration: 06m 48s) |
[production] |
| 20:19 |
<milimetric@deploy1001> |
Started deploy [analytics/refinery@d806b62]: Update jar versions on modified jobs |
[production] |
| 20:07 |
<onimisionipe@deploy1001> |
deploy aborted: Updating maps1002 to reflect latest changes (duration: 00m 01s) |
[production] |
| 20:07 |
<onimisionipe@deploy1001> |
Started deploy [kartotherian/deploy@e847e7b] (stretch): Updating maps1002 to reflect latest changes |
[production] |
| 20:06 |
<oblivian@puppetmaster1001> |
conftool action : set/pooled=false; selector: dnsdisc=kartotherian,name=eqiad |
[production] |
| 20:06 |
<volans> |
running cumin 'P{O:cache::upload} and A:eqiad' 'run-puppet-agent' |
[production] |
| 20:03 |
<gehel> |
running nodetool repair on system_auth for maps / eqiad servers |
[production] |
| 19:30 |
<arturo> |
T214299 additional reboot for cloudnet1003 |
[production] |