2301-2350 of 4788 results (22ms)
2020-03-03 §
11:56 <joal> Kill actor-hourly oozie test jobs (precision of previous message) [analytics]
11:55 <joal> Kill actor-hourly tests [analytics]
10:50 <elukey> restarted kafka jumbo (kafka + mirror maker) for openjdk upgrades [analytics]
09:22 <joal> Rerunning failed mediawiki-history jobs for 2020-02 after mediawiki-history-denormalize issue [analytics]
09:16 <joal> Manually restarting mediawiki-history-denormalize with new patch to try [analytics]
08:36 <elukey> roll restart kafka-jumbo for openjdk upgrades [analytics]
08:34 <elukey> re-enable timers on an-coord1001 after maintenance [analytics]
08:30 <joal> Correct previsou message: Kill mediawiki-history (not mediawiki-history-reduced) as it is failing [analytics]
08:30 <joal> Kill mediawiki-history-reduced as it is failing [analytics]
08:22 <elukey> hive metastore/server2 now running without zookeeper settings and without DBTokenStore (in memory one used instead, the default) [analytics]
08:19 <elukey> restart oozie/hive daemons on an-coord1001 for openjdk upgrades [analytics]
06:41 <elukey> roll restart druid daemons for openjdk upgrades [analytics]
06:39 <elukey> sto timers on an-coord1001 to facilitate daemon restarts (hive/oozie) [analytics]
2020-03-02 §
19:58 <joal> Remove faulty _REFINED file at /wmf/data/event/mediawiki_revision_score/datacenter=eqiad/year=2020/month=3/day=2/hour=10/_REFINED [analytics]
15:38 <elukey> apply new settings to all stat/notebooks [analytics]
15:31 <elukey> setting new user.slice global memory/cpu settings on notebook1003 [analytics]
15:25 <elukey> setting new user-slice global memory/cpu settings on stat1007 [analytics]
2020-02-28 §
19:10 <milimetric> deployed 0.0.116 and restarted webrequest load bundle at 2020-02-28T14 [analytics]
14:49 <joal> Drop test keyspaces in cassandra cluster [analytics]
2020-02-27 §
21:16 <milimetric> tried to deploy AQS but it failed with the same integration test on mediarequests, sending email [analytics]
2020-02-26 §
15:06 <ottomata> dropped and re-added backfilled partitions on event.CentralNoticeImpression table to propogate schema alter on main table - T244771 [analytics]
09:50 <joal> Force delete old api/cirrus events from HDFS trash to free some space [analytics]
2020-02-24 §
18:20 <elukey> move report updater jobs from stat1007 to an-launcher1001 [analytics]
2020-02-22 §
14:21 <elukey> restart hadoop-yarn-nodemanager on analytics1044 - broken disk, apply hiera overrides to exclude it [analytics]
14:11 <elukey> restart hadoop-yarn-nodemanager on analytics1073 - process died, logs saved in /home/elukey [analytics]
2020-02-21 §
16:04 <ottomata> altered event.CentralNoticeImpression table column event.campaignStatuses to type string, will backfill data - T244771 [analytics]
11:49 <elukey> restart varnishkafka on various cp30xx nodes [analytics]
11:41 <elukey> restart varnishkafka on cp3057 (stuck in timeouts to kafka, analytics alarms raised) [analytics]
08:19 <fdans> deploying refinery [analytics]
00:11 <joal> Rerun failed wikidata-json_entity-weekly-coord instances after having created the missing hive table [analytics]
2020-02-20 §
16:57 <fdans> refinery source jars updated [analytics]
16:39 <fdans> deploying refinery source 0.0.114 [analytics]
15:16 <fdans> deploying AQS [analytics]
2020-02-19 §
16:58 <ottomata> Deployed refinery using scap, then deployed onto hdfs [analytics]
2020-02-17 §
18:29 <elukey> reboot turnilo and superset's hosts for kernel upgrades [analytics]
18:25 <elukey> restart kafka on kafka-jumbo1001 to pick up new openjdk updates [analytics]
18:22 <elukey> restart cassandra on aqs1004 to pick up new openjdk updates [analytics]
17:59 <elukey> restart druid daemons on druid1003 to pick up new openjdk updates [analytics]
17:58 <elukey> restart cassandra on aqs1004 to pick up new openjdk updates [analytics]
17:56 <elukey> restart hadoop daemons on analytics1042 to pick up new openjdk updates [analytics]
2020-02-15 §
12:07 <elukey> re-run failed pageview druid hour [analytics]
12:05 <elukey> re-run failed virtualpageview hours [analytics]
2020-02-12 §
14:33 <elukey> restart hue on analytics-tool1001 [analytics]
13:36 <joal> Kill-restart webrequest bundle to see if it mitigates the error [analytics]
2020-02-10 §
15:26 <elukey> kill application_1576512674871_246621 (consuming too much memory) [analytics]
14:31 <elukey> kill application_1576512674871_246419 (eating a ton of ram on the cluster) [analytics]
2020-02-08 §
09:35 <elukey> created /wmf/data/raw/wikidata/dumps/all_ttl on hdfs [analytics]
09:35 <elukey> created /wmf/data/raw/wikidata/dumps/all_json on hdfs [analytics]
2020-02-05 §
21:14 <joal> Kill data_quality_stats-hourly-bundle and data_quality_stats-daily-bundle [analytics]
21:11 <joal> Kill-restart mediawiki-history-dumps-coord, drop existing data, and restart at 2019-11 [analytics]