|
2018-05-29
§
|
| 07:56 |
<moritzm> |
upgrading mw1276-mw1290 to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) |
[production] |
| 07:49 |
<elukey> |
reimage druid1002 to debian stretch |
[production] |
| 07:47 |
<gilles@tin> |
Synchronized wmf-config/InitialiseSettings.php: T187299 Launch performance survey on ruwiki (duration: 01m 50s) |
[production] |
| 07:36 |
<hashar> |
cleaning up some caches on castor02.integration.eqiad.wmflabs (under /srv/jenkins-workspace/caches/ ) |
[releng] |
| 07:26 |
<moritzm> |
upgrading remaining app servers in eqiad to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) |
[production] |
| 07:05 |
<elukey> |
re-run webrequest-load-wf-text-2018-5-29-1 |
[analytics] |
| 06:52 |
<elukey> |
roll restart hadoop master daemons to pick up the new zookeeper settings |
[production] |
| 06:12 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/435961 |
[releng] |
| 05:20 |
<marostegui> |
Restart MySQL on db2045 (s8 codfw master) - T195598 |
[production] |
| 05:13 |
<marostegui> |
Stop MySQL on db2094 and db2095 for testing - T190704 |
[production] |
| 04:47 |
<legoktm> |
building https://gerrit.wikimedia.org/r/430037 |
[releng] |
| 04:12 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue May 29 04:12:10 UTC 2018 (duration 14m 32s) |
[production] |
| 03:57 |
<l10nupdate@tin> |
scap sync-l10n completed (1.32.0-wmf.5) (duration: 14m 29s) |
[production] |
| 02:59 |
<l10nupdate@tin> |
scap sync-l10n completed (1.32.0-wmf.4) (duration: 13m 18s) |
[production] |
| 01:57 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/435952 |
[releng] |
| 01:34 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/435950 |
[releng] |
| 01:31 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/435949 |
[releng] |
| 01:27 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/435948 |
[releng] |
| 01:16 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/435706 https://gerrit.wikimedia.org/r/435707 https://gerrit.wikimedia.org/r/434799 |
[releng] |
| 01:02 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/435946 |
[releng] |
| 01:00 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/435945 |
[releng] |
| 00:44 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/435943 |
[releng] |
| 00:14 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/435939 |
[releng] |
| 00:11 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/435938 |
[releng] |
|
2018-05-28
§
|
| 20:14 |
<twentyafterfour> |
Test failures on https://gerrit.wikimedia.org/r/#/c/435825/ are preventing deployment of the fix for a critical deployment blocker (see T195514) 1.32.0-wmf.5 still blocked refs T191051 |
[production] |
| 20:10 |
<twentyafterfour> |
train still held up by test failures: https://gerrit.wikimedia.org/r/#/c/435825/ |
[production] |
| 20:02 |
<elukey> |
restart kafka on kafka1003 as attempt to solve the under-replicated partitions warning |
[production] |
| 19:22 |
<twentyafterfour@tin> |
Synchronized php-1.32.0-wmf.5/extensions/CentralNotice/: sync wmf.5 CentralNotice for AndyRussG (duration: 01m 25s) |
[production] |
| 19:12 |
<elukey> |
roll restart of kafka-mirror maker (main eqiad -> jumbo) on kafka-jumbo* for zookeeper conf updates |
[production] |
| 19:07 |
<twentyafterfour> |
attempting to get the wmf.5 train back on track. Deploying a fix for T195514 (https://gerrit.wikimedia.org/r/c/435292/) to unblock T191051 |
[production] |
| 18:51 |
<elukey> |
rerun webrequest-load-wf-upload-2018-5-28-14 |
[analytics] |
| 18:16 |
<elukey> |
restart kafka mirror maker on kafka1012->14 - failed after the last round of kafka restarts |
[analytics] |
| 18:16 |
<elukey> |
restart kafka mirror maker on kafka1012->14 - failed after the last round of kafka restarts |
[production] |
| 17:26 |
<elukey> |
roll restart of kafka on kafka-jumbo* to pick up the new zookeeper settings |
[production] |
| 17:20 |
<gehel@tin> |
Finished deploy [wdqs/wdqs@0e40344]: WDQS updater and GUI (duration: 08m 59s) |
[production] |
| 17:19 |
<elukey> |
restart kafka on kafka1012->23 to pick up the new zookeeper settings |
[production] |
| 17:11 |
<gehel@tin> |
Started deploy [wdqs/wdqs@0e40344]: WDQS updater and GUI |
[production] |
| 17:08 |
<moritzm> |
upgrading mwdebug servers in codfw to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) |
[production] |
| 16:48 |
<moritzm> |
upgrading codfw video scalers to hhvm-wikidiff 1.7.0 |
[production] |
| 16:31 |
<elukey> |
roll restart kafka on kafka100[1-3] to pick up new zookeeper settings |
[production] |
| 16:21 |
<elukey> |
zookeeper cluster restart completed (main-eqiad / conf1*) |
[production] |
| 16:18 |
<elukey> |
stop and mask zookeeper on conf1002 |
[production] |
| 16:16 |
<elukey> |
restart prometheus-burrow-exporter on kafkamon* |
[production] |
| 16:12 |
<moritzm> |
upgrading job runners in codfw to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) |
[production] |
| 16:02 |
<marostegui> |
Stop MySQL on db2095 for testing - T190704 |
[production] |
| 15:59 |
<elukey> |
swap zookeeper from conf1002 to conf1005 |
[production] |
| 15:56 |
<marostegui> |
Reboot db1124 and db1125 for more testing - T190704 |
[production] |
| 15:49 |
<_joe_> |
uploading cergen 0.2.3 |
[production] |
| 15:08 |
<Lucas_WMDE> |
wikidata-constraints update MediaWiki and extensions to current master |
[wikidata-dev] |
| 14:44 |
<Amir1> |
EU SWAT is done |
[production] |