4651-4700 of 5027 results (33ms)
2016-08-08 §
18:30 <ottomata> restarting kafka broker on kafka1013 to test eventlogging leader rebalances [analytics]
15:13 <ottomata> deploying eventlogging/analytics - kafka-python 1.3.0 for both consumers and producers [analytics]
14:13 <joal> Loading 2016-06 in clean new aqs [analytics]
14:09 <joal> Adding test data onto newly wiped aqs cluster [analytics]
14:06 <joal> Updating cassandra compaction to deflate on newly wiped cluster [analytics]
2016-08-05 §
15:39 <joal> Restart oozie jobs for druid loading from production refinery instead of joal [analytics]
14:31 <joal> Retrying deploying refinery from scap [analytics]
13:51 <joal> Stopping pagecounts-[raw|all-sites] oozie jobs (load and archive) [analytics]
13:07 <joal> Deploying refinery using scap [analytics]
12:59 <joal> Rolled back refinery interactive deploy [analytics]
12:54 <joal> Deploy refinery using brand new scap deploy ! [analytics]
07:42 <elukey> ran apt-get clean on analytics1027 to free space [analytics]
2016-08-04 §
19:50 <ottomata> now running kafka-python 1.2.5 for eventlogging-service-eventbus in codfw, removed downtime for kafka200[12] [analytics]
17:36 <elukey> added the analytics-deploy key to the Keyholder for the Analytics Refinery scap3 migration (also updated https://wikitech.wikimedia.org/wiki/Keyholder) [analytics]
17:28 <elukey> deploying the refinery with scap3 for the first time on all nodes [analytics]
2016-07-29 §
01:55 <milimetric> limn1 disk full, no idea how to clean it because /public refuses to list its files or listen to me when I try to delete it [analytics]
2016-07-28 §
17:37 <ottomata> powercycling analytics1032 [analytics]
2016-07-26 §
10:13 <joal> Re-deploying refinery after bug fix [analytics]
09:26 <joal> Deploying refinery [analytics]
08:41 <joal> Deploying refinery-source using Jenkins [analytics]
2016-07-25 §
18:31 <ottomata> upgrading kafka to 0.9 in main-codfw, first kafka2001 then 2002 [analytics]
2016-07-20 §
19:40 <joal> Relaunch 2016-07-19 cassandra per-article-daily oozie job [analytics]
15:45 <elukey> executed https://phabricator.wikimedia.org/P3520 on aqs100[456] for both a/b cassandra instances [analytics]
15:32 <elukey> raising compaction throughput to 256 on aqs100[456] [analytics]
2016-07-18 §
17:16 <joal> Change compression from lz4 to deflate on aqs100[456] [analytics]
17:16 <joal> Change compression from lz4 to deflate [analytics]
08:59 <joal> deploy restabase on aqs100[23] [analytics]
08:36 <elukey> re-executed cassandra-daily-wf-local_group_default_T_pageviews_per_article_flat-2016-7-16 (failed oozie job) [analytics]
2016-07-15 §
15:29 <ottomata> restarting hadoop-mapreduce-historyserver to apply yarn log aggreation retention settings [analytics]
2016-07-14 §
20:02 <ottomata> restarting hadoop-yarn-resourcemanager on analytics1002 and then analytics1001 to apply yarn log aggregation change [analytics]
2016-07-13 §
13:45 <ottomata> restarting hadoop nodemanagers to apply log aggregation retention check interval change [analytics]
13:14 <elukey> varnishkafka upgraded from 1.0.10-1 to 1.0.11-1 manually on cp3008.esams (misc) and via apt for the whole cache maps cluster [analytics]
09:05 <joal> Deploying refinery to HDFS [analytics]
08:59 <joal> deploying refinery from tin [analytics]
2016-07-12 §
17:41 <joal> Insert test data in aqs100[456] to prevent false alarms [analytics]
13:05 <ottomata> restarting nodemanagers on analytics 1039 1046 and 1054 [analytics]
2016-07-11 §
20:31 <ottomata> rolling restart of hadoop-yarn-nodemanager to apply log aggregation retention seconds [analytics]
11:33 <joal> Deploying aqs on aqs100[456] (new cluster, no traffic) [analytics]
11:22 <joal> Succesfull deployment in beta - Deploying aqs on aqs1001 as canary [analytics]
11:18 <joal> deploying aqs on deployment-prep [analytics]
2016-07-04 §
20:38 <joal> Insert monitoring test data into cassandra on hosts aqs100[456] to prevent icinga alarms [analytics]
20:38 <joal> Insert manitoring testto make tests pass [analytics]
2016-06-22 §
15:01 <elukey> rebooting bohrium.eqiad.wmnet (running piwik) for kernel upgrades [analytics]
2016-06-15 §
08:34 <joal> Restart misc load job with 10% data loss error threshold [analytics]
2016-06-09 §
14:37 <elukey> Tested retention.bytes=2G for kafka webrequest_misc [analytics]
14:36 <elukey> Tested retention.bytes=2G for kafka webrequest_misc - setting removed [analytics]
2016-06-08 §
18:11 <elukey> removed retention.bytes override configuration for kafka webrequest_text (didn't work) [analytics]
16:03 <elukey> temporary set a 10TB upperbound to the Kafka webrequest_text topic to free space [analytics]
08:45 <elukey> removed temporary retention override for kafka webrequest_text topic (T136690) [analytics]
08:17 <elukey> lowering down webrequest_text kafka topic retention time from 7 days to 4 days to free disk space [analytics]