2301-2350 of 4778 results (10ms)
2020-03-03 §
08:19 <elukey> restart oozie/hive daemons on an-coord1001 for openjdk upgrades [analytics]
06:41 <elukey> roll restart druid daemons for openjdk upgrades [analytics]
06:39 <elukey> sto timers on an-coord1001 to facilitate daemon restarts (hive/oozie) [analytics]
2020-03-02 §
19:58 <joal> Remove faulty _REFINED file at /wmf/data/event/mediawiki_revision_score/datacenter=eqiad/year=2020/month=3/day=2/hour=10/_REFINED [analytics]
15:38 <elukey> apply new settings to all stat/notebooks [analytics]
15:31 <elukey> setting new user.slice global memory/cpu settings on notebook1003 [analytics]
15:25 <elukey> setting new user-slice global memory/cpu settings on stat1007 [analytics]
2020-02-28 §
19:10 <milimetric> deployed 0.0.116 and restarted webrequest load bundle at 2020-02-28T14 [analytics]
14:49 <joal> Drop test keyspaces in cassandra cluster [analytics]
2020-02-27 §
21:16 <milimetric> tried to deploy AQS but it failed with the same integration test on mediarequests, sending email [analytics]
2020-02-26 §
15:06 <ottomata> dropped and re-added backfilled partitions on event.CentralNoticeImpression table to propogate schema alter on main table - T244771 [analytics]
09:50 <joal> Force delete old api/cirrus events from HDFS trash to free some space [analytics]
2020-02-24 §
18:20 <elukey> move report updater jobs from stat1007 to an-launcher1001 [analytics]
2020-02-22 §
14:21 <elukey> restart hadoop-yarn-nodemanager on analytics1044 - broken disk, apply hiera overrides to exclude it [analytics]
14:11 <elukey> restart hadoop-yarn-nodemanager on analytics1073 - process died, logs saved in /home/elukey [analytics]
2020-02-21 §
16:04 <ottomata> altered event.CentralNoticeImpression table column event.campaignStatuses to type string, will backfill data - T244771 [analytics]
11:49 <elukey> restart varnishkafka on various cp30xx nodes [analytics]
11:41 <elukey> restart varnishkafka on cp3057 (stuck in timeouts to kafka, analytics alarms raised) [analytics]
08:19 <fdans> deploying refinery [analytics]
00:11 <joal> Rerun failed wikidata-json_entity-weekly-coord instances after having created the missing hive table [analytics]
2020-02-20 §
16:57 <fdans> refinery source jars updated [analytics]
16:39 <fdans> deploying refinery source 0.0.114 [analytics]
15:16 <fdans> deploying AQS [analytics]
2020-02-19 §
16:58 <ottomata> Deployed refinery using scap, then deployed onto hdfs [analytics]
2020-02-17 §
18:29 <elukey> reboot turnilo and superset's hosts for kernel upgrades [analytics]
18:25 <elukey> restart kafka on kafka-jumbo1001 to pick up new openjdk updates [analytics]
18:22 <elukey> restart cassandra on aqs1004 to pick up new openjdk updates [analytics]
17:59 <elukey> restart druid daemons on druid1003 to pick up new openjdk updates [analytics]
17:58 <elukey> restart cassandra on aqs1004 to pick up new openjdk updates [analytics]
17:56 <elukey> restart hadoop daemons on analytics1042 to pick up new openjdk updates [analytics]
2020-02-15 §
12:07 <elukey> re-run failed pageview druid hour [analytics]
12:05 <elukey> re-run failed virtualpageview hours [analytics]
2020-02-12 §
14:33 <elukey> restart hue on analytics-tool1001 [analytics]
13:36 <joal> Kill-restart webrequest bundle to see if it mitigates the error [analytics]
2020-02-10 §
15:26 <elukey> kill application_1576512674871_246621 (consuming too much memory) [analytics]
14:31 <elukey> kill application_1576512674871_246419 (eating a ton of ram on the cluster) [analytics]
2020-02-08 §
09:35 <elukey> created /wmf/data/raw/wikidata/dumps/all_ttl on hdfs [analytics]
09:35 <elukey> created /wmf/data/raw/wikidata/dumps/all_json on hdfs [analytics]
2020-02-05 §
21:14 <joal> Kill data_quality_stats-hourly-bundle and data_quality_stats-daily-bundle [analytics]
21:11 <joal> Kill-restart mediawiki-history-dumps-coord, drop existing data, and restart at 2019-11 [analytics]
21:06 <joal> Kill-restart mediawiki-wikitext-history-coord and mediawiki-wikitext-current-coord [analytics]
20:51 <joal> Deploy refinery using scap [analytics]
20:29 <joal> Refinery-source released in archiva by jenkins [analytics]
20:20 <joal> Deploy hdfs-tools 0.0.5 using scap [analytics]
2020-02-03 §
11:20 <elukey> restart oozie on an-coord1001 [analytics]
10:11 <elukey> enable all timers on an-coord1001 after spark encryption/auth settings [analytics]
09:32 <elukey> roll restart yarn node managers again to pick up spark encryption/authentication settings [analytics]
08:34 <elukey> stop timers on an-coord1001 to drain the cluster and ease the deploy of spark encryption settings [analytics]
07:58 <elukey> roll restart hadoop yarn node managers to pick up new libcrypto.so link (shouldn't be necessary but just in case) [analytics]
07:24 <elukey> create /usr/lib/x86_64-linux-gnu/libcrypto.so on all the analytics nodes via puppet [analytics]