2801-2850 of 5926 results (25ms)
2020-11-05 §
14:55 <elukey> shutdown kafka-jumbo1001 to swap NICs (1g -> 10g) [analytics]
06:30 <elukey> truncate application_1601916545561_129457's taskmanager.log (~600G) on an-worker1113 due to partition 'e' full [analytics]
02:05 <milimetric> deployed refinery pointing to refinery-source v0.0.138 [analytics]
2020-11-04 §
09:20 <elukey> upgrade hue to 4.8.0 on hue-next [analytics]
2020-11-03 §
16:52 <elukey> mv /srv/analytics.wikimedia.org/published/datasets/archive/public-datasets to /srv/backup/public-datasets on thorium - T265971 [analytics]
15:52 <elukey> re-enable timers after maintenance [analytics]
14:02 <elukey> stop timers on an-launcher1002 to drain the cluster (an-coord1001 maintenance prep-step) [analytics]
13:02 <elukey> force a restart of performance-asoranking.service on stat1007 after fix for pandas' sort() - T266985 [analytics]
07:26 <elukey> re-run cassandra-daily-coord-local_group_default_T_pageviews_per_article_flat failed hour via hue [analytics]
2020-11-02 §
21:15 <ottomata> evolved Hive table event.contenttranslationabusefilter to match migrated event platform schema - T259163 [analytics]
13:40 <elukey> roll restart zookeeper ok an-conf* to pick up new openjdk upgrades [analytics]
12:40 <elukey> forced re-creation of base jupyterhub venvs on stat1007 [analytics]
2020-10-30 §
17:01 <elukey> kafka preferred-replica-election on jumbo1001 [analytics]
2020-10-29 §
14:25 <elukey> restart zookeeper on an-conf1001 for openjdk upgrades [analytics]
2020-10-27 §
17:38 <ottomata> restrict Fuzz Faster U Fool user agents from submittnig eventlogging legacy systemd data - T266130 [analytics]
2020-10-22 §
14:05 <ottomata> bump camus version to wmf12 for all camus jobs. should be no-op now. - T251609 [analytics]
13:56 <ottomata> camus-eventgate-main_events now uses EventStreamConfig to discover topics to ingest, but still uses regex to find topics to monitor - T251609 [analytics]
13:04 <ottomata> camus-eventgate-analytics_events now uses EventStreamConfig to discovery topics to ingest and canary topics to monitor - T251609 [analytics]
13:03 <elukey> restart turnilo to pick up new wmf_netflow settings [analytics]
11:51 <ottomata> camus-eventgate-analytics-external now uses EventStreamConfig to discovery topics to ingest and canary topics to monitor [analytics]
07:03 <elukey> decom analytics1057 from the Hadoop cluster [analytics]
06:54 <elukey> restart httpd on matomo1002, errors while connecting [analytics]
06:31 <elukey> restart turnilo to apply new settings for wmf_netflow [analytics]
06:06 <elukey> execute "sudo -u hdfs kerberos-run-command hdfs hdfs dfs -chown -R analytics /wmf/data/archive/geoip" on an-launcher1002 - permission issues for 'analytics' and /wmf/data/archive/geoip [analytics]
02:37 <ottomata> re-run webrequest-load-wf-{text,upload}-2020-10-21-{19,20} oozie jobs after they timed out waiting for data due to camus misconfiguration (fixed in https://gerrit.wikimedia.org/r/c/operations/puppet/+/635678) [analytics]
2020-10-21 §
20:12 <razzi> stop nginx on analytics-tool1001.eqiad.wmnet to switch to envoy (hue-next) [analytics]
20:10 <razzi> stop nginx on analytics-tool1001.eqiad.wmnet to switch to envoy (hue) [analytics]
20:07 <razzi> stop nginx on analytics-tool1007.eqiad.wmnet to switch to envoy (turnilo) [analytics]
20:05 <razzi> stop nginx on analytics-tool1004.eqiad.wmnet to switch to envoy (superset) [analytics]
20:02 <razzi> stop nginx on matomo1002.eqiad.wmnet to switch to envoy [analytics]
10:41 <elukey> decommission analytics1052 from the hadoop cluster [analytics]
10:26 <elukey> move journalnode from analytics1052 (to be decommed) to an-worker1080 [analytics]
2020-10-20 §
20:59 <mforns> Deploying refinery with refinery-deploy-to-hdfs (for 0.0.137) [analytics]
20:24 <mforns> Deploying refinery with scap for v0.0.137 [analytics]
20:00 <mforns> Deployed refinery-source v0.0.137 [analytics]
15:00 <ottomata> disabling sending EventLogging events to eventlogging-valid-mixed topic - T265651 [analytics]
13:34 <elukey> upgrade superset's presto TLS config after the above changes [analytics]
13:33 <elukey> move presto to pupet host TLS certificates [analytics]
10:29 <klausman> rocm38 install on an-worker1101 successful, rebooting to make sure everything is in place [analytics]
06:41 <elukey> decom analytics1056 from the hadoop cluster [analytics]
2020-10-19 §
14:40 <ottomata> restarted eventlogging-processor with filter to skip events already migrated to event platform - T262304 [analytics]
10:09 <elukey> add pps/bps measures to wmf_netflow in turnilo [analytics]
07:27 <elukey> decom analytics1055 from the hadoop cluster [analytics]
06:47 <elukey> turnilo upgraded to 1.27.0 [analytics]
2020-10-18 §
07:01 <elukey> decom analytics1054 from hadoop [analytics]
2020-10-17 §
06:08 <elukey> decom analytics1053 from the hadoop cluster [analytics]
2020-10-15 §
17:57 <razzi> taking yarn.wikimedia.org offline momentarily to test new tls configuration: T240439 [analytics]
14:51 <elukey> roll restart druid-historical daemons on druid1004-1008 to pick up new conn pooling changes [analytics]
07:03 <elukey> restart oozie to pick up the analytics team's admin list [analytics]
06:09 <elukey> decommission analytics1050 from the hadoop cluster [analytics]