1-50 of 945 results (10ms)
2017-11-08 §
15:16 <ottomata> deploying eventlogging analytics change for eventcapsule schema fixes, will be no-op until we deploy puppet changes too [analytics]
11:28 <elukey> resumed cassandra-coord-pageview-per-project-hourly after maintenance to aqs hosts [analytics]
10:04 <elukey> suspended cassandra-coord-pageview-per-project-hourly as prep step to reboot aqs nodes - T179943 [analytics]
2017-11-06 §
15:37 <milimetric> found geowiki was hitting the wrong databases, updated it to always hit analytics-store [analytics]
2017-11-03 §
10:55 <joal> Kill mediawiki-history oozie job to prevent computing october snapshot before fixing reconstruction process [analytics]
2017-11-02 §
08:54 <elukey> relaunched failed pageview-druid-hourly jobs - Druid indexation check failures in the logs (01 Nov 2017 21:00:00 and 01 Nov 2017 19:00:00) [analytics]
2017-11-01 §
20:06 <ottomata> rerunning pageview-druid-hourly-wf-2017-11-1-18 [analytics]
19:05 <ottomata> deploying refinery with refinery/source 0.0.54 for JsonRefine job T162610 [analytics]
18:40 <ottomata> rerunning unique_devices-per_project_family-druid-monthly-wf-2017-10 [analytics]
2017-10-30 §
10:12 <elukey> added Francisco to the analytics-alerts@ mailing list [analytics]
2017-10-27 §
07:40 <elukey> re-run wikidata-articleplaceholder_metrics-wf-2017-10-26 [analytics]
07:36 <elukey> stop & mask hadoop-httpfs.service on analytics1001 after https://gerrit.wikimedia.org/r/#/c/386684/ [analytics]
2017-10-26 §
16:58 <ottomata> now mirroring main Kafka cluster topics to jumbo Kafka cluster,  with MirrorMaker instances running on analytics-eqiad broker nodes. https://phabricator.wikimedia.org/T177216 [analytics]
2017-10-25 §
13:32 <elukey> restart yarn nodemanager and hdfs datanode on analytics1030 to apply new JVM settings [analytics]
2017-10-24 §
20:29 <nuria_> started unique_devices-per_project_family-druid-daily-coord 0102816-170829140538136-oozie-oozi-C [analytics]
20:24 <nuria_> restarted job unique_devices-per_project_family-druid-monthly-coord 0102799-170829140538136-oozie-oozi-C [analytics]
20:23 <nuria_> restarted job uniques-monthly-per-domain-druid 0102785-170829140538136-oozie-oozi-C [analytics]
19:44 <nuria_> killing druid coordinators uniques-monthly and per-project-family: 0066771-170829140538136-oozie-oozi-C,0066767-170829140538136-oozie-oozi-C,0010139-170621131133576-oozie-oozi-C [analytics]
2017-10-23 §
18:50 <joal> Deploying AQS after fix [analytics]
13:30 <joal> deploy AQS from tin [analytics]
2017-10-19 §
20:04 <mforns> Deployed refinery using scap, then deployed onto hdfs [analytics]
11:44 <joal> deploying AQS in beta [analytics]
11:44 <joal> deploying AQS in b [analytics]
2017-10-16 §
17:32 <mforns> restarted EventLogging for changes in blacklist to take effect [analytics]
16:27 <joal> Re-Deploy AQS after monitoring fix [analytics]
16:14 <joal> Deploy AQS with new code [analytics]
2017-10-13 §
16:49 <ottomata> deployed refinery to use rand() for webrequest sampling [analytics]
2017-10-12 §
15:40 <elukey> run kafka preferred-replica-election to allow kafka1013 to re-join the topic leaders [analytics]
14:48 <elukey> disable httpfs access on analytics1001 [analytics]
2017-10-09 §
18:28 <ottomata> resuming oozie druid indexing jobs, 1004-1006 are offline [analytics]
16:34 <ottomata> stopping druid services on druid1006 [analytics]
16:05 <ottomata> pausing all druid oozie coordinators in preperation for druid public separation [analytics]
12:47 <joal> Kill restart oozie job lading mediawiki-history into druid [analytics]
12:14 <joal> Kill-Restart oozie jobs loading banner data into druid [analytics]
12:04 <joal> Deploy refinery onto HDFS [analytics]
11:47 <joal> Deploying refinery from scap [analytics]
08:53 <joal> Rerunning wikidata-articleplaceholder_metrics-wf-2017-10-7 after failure [analytics]
2017-10-06 §
11:10 <elukey> restart all druid daemons to pick up new logging changes [analytics]
11:08 <joal> Rerun pageview-druid-hourly-wf-2017-10-6-9 [analytics]
09:31 <elukey> restart all the druid daemons on druid1005 to apply the new logging rules [analytics]
08:49 <elukey> restarted all the druid broker daemons to pick up the new logging changes [analytics]
2017-10-05 §
13:48 <milimetric> restarted banner_activity-druid-monthly for September again [analytics]
2017-10-04 §
18:39 <ottomata> druid-analytics.svc.eqiad.wmnet:8082 should only be accessible to analytics networks [analytics]
17:32 <ottomata> deploying new LVS service for druid-analytics-broker [analytics]
2017-10-03 §
14:50 <milimetric> restarted failed workflow 0057215-170829140538136-oozie-oozi-W (druid monthly banner activity) [analytics]
2017-09-28 §
10:02 <elukey> renabled camus after maintenance [analytics]
09:51 <elukey> restart mapreduce history server on an1001 to apply new heap settings (Xmx/s to 4g) [analytics]
2017-09-27 §
15:18 <joal> Kill/restart stuck jobs [analytics]
14:45 <elukey> rolling restart of all the Yarn nodemanager daemons on analytics1028-1068 (ease heap consumption pressure, seamless restart) [analytics]
13:40 <elukey> manual failover of HDFS namenode from an1002 to an1001 [analytics]