351-400 of 3496 results (17ms)
2020-11-16 §
08:01 <joal> Repair wmf.webrequest hive table partitions [analytics]
08:01 <joal> Recreate wmf.webrequest hive table with new partitioning [analytics]
08:00 <joal> Drop webrequest table [analytics]
07:55 <joal> Kill webrequest-bundle oozie job for table update [analytics]
2020-11-15 §
08:27 <elukey> truncate -s 10g /var/lib/hadoop/data/n/yarn/logs/application_1601916545561_173219/container_e25_1601916545561_173219_01_000177/stderr on an-worker1100 [analytics]
08:21 <elukey> sudo truncate -s 10g /var/lib/hadoop/data/c/yarn/logs/application_1601916545561_173219/container_e25_1601916545561_173219_01_000019/stderr on an-worker1098 [analytics]
2020-11-10 §
19:32 <joal> Deploy wikistats2 v2.8.2 [analytics]
18:16 <joal> Releasing refinery-source v0,0,139 to archiva [analytics]
14:48 <mforns> restarted data quality stats daily bundle with new metric [analytics]
13:30 <elukey> add hive-server2 to an-coord1002 [analytics]
07:40 <elukey> upgrade hue to hue_4.8.0-2 on an-tool1009 [analytics]
2020-11-09 §
18:34 <elukey> drop hdfs-balancer multi-gb log file from launcher1002 [analytics]
18:33 <elukey> manually start logrotate.timer apt.timer etc.. on an-launcher1002 - stopped since the last time that I have disabled timers [analytics]
17:48 <razzi> reboot an-coord1002 to see if it updates kernel cpu instructions [analytics]
2020-11-08 §
06:31 <elukey> truncate huge log file on an-worker1103 for app id application_1601916545561_147041 [analytics]
2020-11-06 §
19:00 <mforns> launched backfilling of data quality stats for os_family_entropy_by_access_method [analytics]
2020-11-05 §
18:32 <razzi> shutting down kafka-jumbo1005 to allow dcops to upgrade NIC [analytics]
17:47 <razzi> shutting down kafka-jumbo1004 to allow dcops to upgrade NIC [analytics]
16:57 <razzi> shutting down kafka-jumbo1003 to allow dcops to upgrade NIC [analytics]
16:25 <razzi> shutting down kafka-jumbo1002 to allow dcops to upgrade NIC [analytics]
14:55 <elukey> shutdown kafka-jumbo1001 to swap NICs (1g -> 10g) [analytics]
06:30 <elukey> truncate application_1601916545561_129457's taskmanager.log (~600G) on an-worker1113 due to partition 'e' full [analytics]
02:05 <milimetric> deployed refinery pointing to refinery-source v0.0.138 [analytics]
2020-11-04 §
09:20 <elukey> upgrade hue to 4.8.0 on hue-next [analytics]
2020-11-03 §
16:52 <elukey> mv /srv/analytics.wikimedia.org/published/datasets/archive/public-datasets to /srv/backup/public-datasets on thorium - T265971 [analytics]
15:52 <elukey> re-enable timers after maintenance [analytics]
14:02 <elukey> stop timers on an-launcher1002 to drain the cluster (an-coord1001 maintenance prep-step) [analytics]
13:02 <elukey> force a restart of performance-asoranking.service on stat1007 after fix for pandas' sort() - T266985 [analytics]
07:26 <elukey> re-run cassandra-daily-coord-local_group_default_T_pageviews_per_article_flat failed hour via hue [analytics]
2020-11-02 §
21:15 <ottomata> evolved Hive table event.contenttranslationabusefilter to match migrated event platform schema - T259163 [analytics]
13:40 <elukey> roll restart zookeeper ok an-conf* to pick up new openjdk upgrades [analytics]
12:40 <elukey> forced re-creation of base jupyterhub venvs on stat1007 [analytics]
2020-10-30 §
17:01 <elukey> kafka preferred-replica-election on jumbo1001 [analytics]
2020-10-29 §
14:25 <elukey> restart zookeeper on an-conf1001 for openjdk upgrades [analytics]
2020-10-27 §
17:38 <ottomata> restrict Fuzz Faster U Fool user agents from submittnig eventlogging legacy systemd data - T266130 [analytics]
2020-10-22 §
14:05 <ottomata> bump camus version to wmf12 for all camus jobs. should be no-op now. - T251609 [analytics]
13:56 <ottomata> camus-eventgate-main_events now uses EventStreamConfig to discover topics to ingest, but still uses regex to find topics to monitor - T251609 [analytics]
13:04 <ottomata> camus-eventgate-analytics_events now uses EventStreamConfig to discovery topics to ingest and canary topics to monitor - T251609 [analytics]
13:03 <elukey> restart turnilo to pick up new wmf_netflow settings [analytics]
11:51 <ottomata> camus-eventgate-analytics-external now uses EventStreamConfig to discovery topics to ingest and canary topics to monitor [analytics]
07:03 <elukey> decom analytics1057 from the Hadoop cluster [analytics]
06:54 <elukey> restart httpd on matomo1002, errors while connecting [analytics]
06:31 <elukey> restart turnilo to apply new settings for wmf_netflow [analytics]
06:06 <elukey> execute "sudo -u hdfs kerberos-run-command hdfs hdfs dfs -chown -R analytics /wmf/data/archive/geoip" on an-launcher1002 - permission issues for 'analytics' and /wmf/data/archive/geoip [analytics]
02:37 <ottomata> re-run webrequest-load-wf-{text,upload}-2020-10-21-{19,20} oozie jobs after they timed out waiting for data due to camus misconfiguration (fixed in https://gerrit.wikimedia.org/r/c/operations/puppet/+/635678) [analytics]
2020-10-21 §
20:12 <razzi> stop nginx on analytics-tool1001.eqiad.wmnet to switch to envoy (hue-next) [analytics]
20:10 <razzi> stop nginx on analytics-tool1001.eqiad.wmnet to switch to envoy (hue) [analytics]
20:07 <razzi> stop nginx on analytics-tool1007.eqiad.wmnet to switch to envoy (turnilo) [analytics]
20:05 <razzi> stop nginx on analytics-tool1004.eqiad.wmnet to switch to envoy (superset) [analytics]
20:02 <razzi> stop nginx on matomo1002.eqiad.wmnet to switch to envoy [analytics]