|
2017-02-10
§
|
| 08:41 |
<elukey> |
restarting kafka mirror maker and jmxtrans of kafka[12]00[123] for java security upgrades |
[production] |
| 08:30 |
<marostegui> |
Deploye alter table s3 officewiki.echo_notification and mediawikiwiki.echo_notification tables only on codfw - T136428 |
[production] |
| 04:06 |
<mutante> |
ganglia - switching aggregators from 1001 to 1002 and 2001 to 2002, there might be minor gaps in the graphs, but hey, it's deprecated anyways |
[production] |
| 03:48 |
<mutante> |
icinga - live hack fixing config - due to partially removed decom hosts mc2001-mc2016 |
[production] |
| 02:37 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Fri Feb 10 02:37:59 UTC 2017 (duration 5m 26s) |
[production] |
| 02:32 |
<l10nupdate@tin> |
scap sync-l10n completed (1.29.0-wmf.11) (duration: 11m 53s) |
[production] |
| 01:48 |
<ebernhardson@tin> |
Synchronized wmf-config/CirrusSearch-common.php: Setup sister search prefix display types T149806 (duration: 00m 40s) |
[production] |
| 01:43 |
<ebernhardson@tin> |
Synchronized wmf-config/CirrusSearch-common.php: Setup sister search prefix display types T149806 (duration: 00m 48s) |
[production] |
| 01:01 |
<brion> |
transcode queue back to normal |
[production] |
| 00:53 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:336944|Set $wgPageAssessmentsSubprojects to true on English Wikipedia]] T157654 (duration: 00m 43s) |
[production] |
| 00:53 |
<brion> |
transcode high-prio queue may be briefly blocked by an influx of low-res transcodes queued in bulk. should return to normal in a bit. |
[production] |
| 00:32 |
<thcipriani@tin> |
Synchronized php-1.29.0-wmf.11/extensions/WikimediaEvents: SWAT: [[gerrit:336896|Enable Sister project search AB test]] T149806 (duration: 00m 45s) |
[production] |
|
2017-02-09
§
|
| 22:58 |
<bblack@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=cp2006.codfw.wmnet |
[production] |
| 22:57 |
<bblack> |
cp2006: unresponsive control, powercycled from racadm, normal boot, no evidence in logs - repooling for now |
[production] |
| 22:48 |
<thcipriani@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.29.0-wmf.11 |
[production] |
| 22:44 |
<thcipriani@tin> |
Synchronized php-1.29.0-wmf.11/skins/CologneBlue/SkinCologneBlue.php: [[gerrit:336931|Revert "Remove warning suppression"]] (duration: 00m 59s) |
[production] |
| 22:42 |
<bblack> |
cp2006 depooled due to icinga report of host-down |
[production] |
| 22:42 |
<bblack@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=cp2006.codfw.wmnet |
[production] |
| 22:02 |
<brion> |
brion running tests of requeueTranscodes.php on terbium to restart subsets of video scaler work |
[production] |
| 21:54 |
<mutante> |
cp3014,cp3020,cp3022 - puppet node deactivate - cp3020 delete salt key (T130883) |
[production] |
| 20:45 |
<thcipriani@tin> |
Synchronized php-1.29.0-wmf.11/skins/CologneBlue/SkinCologneBlue.php: [[gerrit:336714|Fix a bunch of undefined indexes]] T157619 (sync actual skin file) (duration: 00m 40s) |
[production] |
| 20:41 |
<thcipriani@tin> |
Synchronized php-1.29.0-wmf.11/skins/CologneBlue/CologneBlue.php: [[gerrit:336714|Fix a bunch of undefined indexes]] T157619 (duration: 00m 41s) |
[production] |
| 20:24 |
<otto@tin> |
Finished deploy [analytics/refinery@9e689f3]: (no justification provided) (duration: 00m 20s) |
[production] |
| 20:23 |
<otto@tin> |
Started deploy [analytics/refinery@9e689f3]: (no justification provided) |
[production] |
| 20:23 |
<otto@tin> |
Started deploy [analytics/refinery@9e689f3]: (no justification provided) |
[production] |
| 20:21 |
<otto@tin> |
Finished deploy [analytics/refinery@9e689f3]: (no justification provided) (duration: 00m 04s) |
[production] |
| 20:21 |
<otto@tin> |
Started deploy [analytics/refinery@9e689f3]: (no justification provided) |
[production] |
| 20:21 |
<otto@tin> |
Started deploy [analytics/refinery@9e689f3]: (no justification provided) |
[production] |
| 20:20 |
<otto@tin> |
Finished deploy [analytics/refinery@9e689f3]: (no justification provided) (duration: 00m 07s) |
[production] |
| 20:20 |
<otto@tin> |
Started deploy [analytics/refinery@9e689f3]: (no justification provided) |
[production] |
| 20:20 |
<otto@tin> |
Started deploy [analytics/refinery@9e689f3]: (no justification provided) |
[production] |
| 20:20 |
<otto@tin> |
Started deploy [analytics/refinery@9e689f3]: (no justification provided) |
[production] |
| 20:19 |
<otto@tin> |
Started deploy [analytics/refinery@9e689f3]: (no justification provided) |
[production] |
| 20:05 |
<thcipriani@tin> |
Synchronized php-1.29.0-wmf.11/resources/src/mediawiki.special/mediawiki.special.search.interwikiwidget.styles.less: SWAT: [[gerrit:336855|Temporary hax to hide cawiki hacked in search sidebar]] T149806 (duration: 00m 40s) |
[production] |
| 20:02 |
<thcipriani@tin> |
Synchronized php-1.29.0-wmf.11/extensions/ConfirmEdit: SWAT: [[gerrit:336864|Add script for counting captchas]] [[gerrit:336866|Use an accurate number of captchas]] (duration: 00m 43s) |
[production] |
| 19:57 |
<ottomata> |
restarting main kafka brokers in codfw and then eqiad to pick up jvm updates |
[production] |
| 19:50 |
<thcipriani@tin> |
Synchronized php-1.29.0-wmf.11/extensions/TimedMediaHandler: SWAT: [[gerrit:336846|TMH job queue split into low and high priority]] PART III T155098 (duration: 00m 44s) |
[production] |
| 19:49 |
<thcipriani@tin> |
Synchronized php-1.29.0-wmf.11/extensions/TimedMediaHandler/TimedMediaHandler.hooks.php: SWAT: [[gerrit:336846|TMH job queue split into low and high priority]] PART II T155098 (duration: 00m 40s) |
[production] |
| 19:47 |
<thcipriani@tin> |
Synchronized php-1.29.0-wmf.11/extensions/TimedMediaHandler/TimedMediaHandler.php: SWAT: [[gerrit:336846|TMH job queue split into low and high priority]] PART I T155098 (duration: 00m 41s) |
[production] |
| 19:37 |
<smalyshev@tin> |
Finished deploy [wdqs/wdqs@1a7cd32]: Deploy new WAR build on 1003 (duration: 00m 16s) |
[production] |
| 19:37 |
<smalyshev@tin> |
Started deploy [wdqs/wdqs@1a7cd32]: Deploy new WAR build on 1003 |
[production] |
| 19:34 |
<smalyshev@tin> |
Finished deploy [wdqs/wdqs@1a7cd32]: Deploy new WAR build on 2003 (duration: 00m 26s) |
[production] |
| 19:34 |
<smalyshev@tin> |
Started deploy [wdqs/wdqs@1a7cd32]: Deploy new WAR build on 2003 |
[production] |
| 19:29 |
<ladsgroup@tin> |
Started deploy [ores/deploy@10fa16b]: (no justification provided) |
[production] |
| 19:28 |
<ladsgroup@tin> |
Started deploy [ores/deploy@a3a410b]: (no justification provided) |
[production] |
| 19:26 |
<thcipriani@tin> |
Synchronized php-1.29.0-wmf.11/extensions/TimedMediaHandler/SpecialTimedMediaHandler.php: SWAT: [[gerrit:336734|Only load necessary fields on Special:TimedMediaHandler lists]] (T157621) (duration: 00m 41s) |
[production] |
| 19:19 |
<ladsgroup@tin> |
Started deploy [ores/deploy@a3a410b]: (no justification provided) |
[production] |
| 19:17 |
<smalyshev@tin> |
Started deploy [wdqs/wdqs@1a7cd32]: Deploy new WAR build |
[production] |
| 19:15 |
<smalyshev@tin> |
Started deploy [wdqs/wdqs@1a7cd32]: (no justification provided) |
[production] |
| 19:14 |
<bd808> |
Restarted logstash on logstash1001. Dead since 2017-02-09T06:39:46 with "java.lang.UnsupportedOperationException" crash in worker thread. |
[production] |