801-850 of 2143 results (17ms)
2018-05-09 §
16:56 <ottomata> disabled 0.9 MirrorMaker on kafka102[023], enabled 1.x MirrorMaker on kafka-jumbo* [analytics]
14:41 <milimetric> finished deploying refinery with proper geoeditors druid indexing template [analytics]
13:59 <ottomata> beginning upgrade of Kafka main-eqiad cluster from 0.9.0.1 to 1.1.0 - T167039 [analytics]
13:49 <milimetric> deploying refinery again, forgot to index a new metric in the new datasource, sorry [analytics]
13:23 <mforns> re-run webrequest-load-wf-misc-2018-5-9-12 via hue [analytics]
13:13 <milimetric> deployed refinery [analytics]
12:58 <milimetric> deploying very simple change just to rename druid datasource [analytics]
12:48 <elukey> re-run webrequest-load-wf-text-2018-5-8-17 via hue [analytics]
2018-05-08 §
20:35 <milimetric> refinery deploy complete [analytics]
20:18 <milimetric> deploying geoeditors for real now [analytics]
20:12 <milimetric> aborting deployment, will deploy data truncation script too [analytics]
20:08 <milimetric> deploying refinery to relaunch geoeditors job [analytics]
17:57 <joal> Mvoe recomputed 2018-03 history snapshot in place of old one (T194075) [analytics]
15:38 <joal> Try again (last time) to rerun mediawiki-history-druid-wf-2018-04 [analytics]
15:06 <ottomata> beginnng Kafka upgrade of main-codfw: T167039 [analytics]
08:01 <elukey> removed cassandra-metrics-collector (graphite) from aqs nodes [analytics]
07:42 <joal> Rerun mediawiki-history-druid-wf-2018-04 in a non-sync way with mediawiki-reduced [analytics]
06:41 <elukey> rolling restart of druid-historicals on druid100[456] due to half of the segments not avaiable [analytics]
2018-05-07 §
12:05 <joal> Rerun mediawiki-history-reduced-wf-2018-04 [analytics]
09:18 <elukey> re-run webrequest-load-wf-text-2018-5-7-7 - failed due to reimages [analytics]
2018-05-04 §
10:11 <elukey> d-[123] Druid cluster upgraded to 0.11 in labs (project analytics) [analytics]
2018-05-03 §
20:29 <milimetric> fixed wikimetrics issues, working fine again [analytics]
19:19 <milimetric> wikimetrics is partly broken until I can figure out what’s going on [analytics]
2018-05-02 §
17:33 <joal> Rerun webrequest-load-wf-text-2018-5-2-15 [analytics]
16:41 <joal> Manually silence pageview-whitelist alarm overwriting /wmf/refinery/current/static_data/pageview/whitelist/whitelist.tsv [analytics]
16:27 <joal> 2018-05-02T14 webrequest dataloss warnings have been checked and are false positives [analytics]
16:17 <joal> Restart oozie mediawiki-history-denormalize job after deploy [analytics]
16:14 <ottomata> bounced eventlogging-consumer@mysql-m4-master-00 after kafka jumbo 1.1.0 upgrade [analytics]
16:05 <joal> Restart oozie webrequest bundle after deploy [analytics]
15:20 <joal> Deploying refinery to hadoop [analytics]
14:45 <joal> Deploying refinery using Scap [analytics]
14:16 <joal> Refinery-source version 0.0.63 finally released to Archiva! [analytics]
13:49 <ottomata> beginning upgrade of kafka-jumbo brokers from 1.0.0 -> 1.1.0 : T193495 [analytics]
13:20 <elukey> restart druid broker on druid100[1-3] to enable druid.sql.enable: true [analytics]
2018-05-01 §
15:33 <elukey> restart historical on druid1003 - exceptions in the logs [analytics]
15:22 <elukey> restart druid-historical on druid1002 - Caused by: java.lang.IllegalArgumentException: Could not resolve type id 'hdfs' into a subtype of [analytics]
11:44 <joal> False positive only in webrequest-load-check_sequence_statistics-wf-upload-2018-5-1-6 [analytics]
07:14 <joal> Rerun webrequest-druid-daily-wf-2018-4-30 [analytics]
06:24 <elukey> roll restart of all middlemanagers on druid100[123] - realtime tasks piled up from hours [analytics]
2018-04-30 §
23:04 <ottomata> blacklisting change-prop and job queue topics from main-eqiad -> analytics (eqiad) [analytics]
22:55 <ottomata> bouncing kafka main-eqiad -> eqiad (analytics) mirror maker [analytics]
19:34 <joal> Retry releasing refinery-source to archiva [analytics]
18:43 <joal> Releasing refinery-source [analytics]
15:53 <joal> Resume webrequest-druid-hourly-coord and pageview-druid-hourly-coord [analytics]
14:23 <joal> Suspend webrequest-druid-hourly-coord and pageview-druid-hourly-coord before druid upgrade [analytics]
14:23 <elukey> disabled cron/check on analytics1003 to respawn banner impressions if needed [analytics]
14:21 <joal> Kill BannerImpressionStream job before upgrading druid [analytics]
2018-04-25 §
14:39 <elukey> re-enable camus after maintenance [analytics]
14:37 <elukey> restart hive-server2 on analytics1003 to pick up settings in https://gerrit.wikimedia.org/r/428919 [analytics]
13:40 <elukey> stop camus on an1003 as prep step to gracefully restart hive server [analytics]