651-700 of 1681 results (9ms)
2018-01-08 §
12:36 <fdans> Deploying AQS [analytics]
12:33 <joal> Update fake-data in cassandra adiing top-by-country needed row [analytics]
11:07 <elukey> re-run webrequest-load-wf-text-2018-1-8-8 (failed after some reboots due to kernel updates) [analytics]
10:07 <elukey> drain + reboot analytics1029,1031->1034 for kernel updates [analytics]
2018-01-07 §
09:01 <elukey> re-enabled puppet on db110[78] - eventlogging_sync restarted on db1108 (analytics-slave) [analytics]
2018-01-06 §
08:09 <elukey> re-enable eventlogging mysql consumers after database maintenance [analytics]
2018-01-05 §
13:18 <fdans> deploying AQS [analytics]
2018-01-04 §
19:54 <joal> Deploying refinery onto hadoop [analytics]
19:45 <joal> Deploy refinery using scap [analytics]
19:38 <joal> Deploy refinery-source using jenkins [analytics]
16:01 <ottomata> killing json_refine_eventlogging_analytics job that started yesterday and has not completed (has no executors running?) application_1512469367986_81514. I think the cluster is just too busy? mw-history job running... [analytics]
10:34 <elukey> re-run mediacounts-archive-wf-2018-01-03 [analytics]
2018-01-03 §
15:00 <ottomata> restarting kafka-jumbo brokers to enable tls version and cipher suite restrictions [analytics]
2018-01-02 §
11:13 <joal> Kill and restart cassandra loading oozie bundle to pick new pageview_top_bycountry job [analytics]
08:22 <elukey> restart druid coordinators to pick up new jvm settings (freeing up 6GB of used memory) [analytics]
2017-12-21 §
15:54 <joal> Start backfilling monthly pageview-by-country [analytics]
15:45 <joal> deploy refinery onto HDFSb [analytics]
15:38 <joal> Deploying refinery with Scap [analytics]
2017-12-20 §
15:40 <ottomata> removing some old webrequest data from hdfs [analytics]
14:56 <ottomata> dropping some old wmf.webrequest partitions and data [analytics]
2017-12-19 §
17:28 <elukey> re-enabled superset [analytics]
17:16 <joal> Initilizaing new cassandra keyspace for pageviews/top-by-country [analytics]
16:52 <elukey> temporarily stop superset to test druid's performances [analytics]
16:34 <elukey> manually started eventlogging cleaner on db1107 to purge/sanitize data up to 90 days ago (tmux is running for user eventlogcleaner) [analytics]
14:10 <elukey> temporary changed JVM Heap settings for the druid broker on druid1001 - Xmx25g Xms10g (run puppet and restart the daemon to rollback) [analytics]
2017-12-12 §
15:54 <milimetric> sieging aqs1004 with 100.000 transactions [analytics]
2017-12-11 §
21:16 <andrewbogott> updating puppetmaster p1.analytics.eqiad.wmflabs to puppet 4 [analytics]
14:07 <elukey> disable druid middlemanager on druid1003 to drain + restart to pick up new logging settings [analytics]
12:59 <elukey> disabled druid middlemanager on druid1002 to drain+restart with new logging config [analytics]
2017-12-08 §
11:15 <joal> Start mediawiki-history oozie jobs new-version [analytics]
10:49 <joal> Update wmf.mediawiki_history as explained in email (rename current table to old, create new one) [analytics]
2017-12-07 §
21:09 <joal> Start clickstream oozie job [analytics]
20:45 <joal> Kill restbase oozie job and restart apis replacing one [analytics]
20:12 <joal> Trying to deploy refinery again [analytics]
19:30 <joal> Deploying refinery now that -source is deployed [analytics]
18:39 <milimetric> Deployed refinery-source using jenkins [analytics]
15:03 <elukey> restart webrequest-misc load job (Dec 7 2017 06:00:00) [analytics]
12:24 <elukey> camus re-enabled after analytics1003 reboot [analytics]
08:31 <elukey> stop camus on an1003 as prep step for reboot [analytics]
2017-12-06 §
14:55 <elukey> restart hue to pick up the new oozie server [analytics]
14:55 <elukey> oozie server accidentally restarted due to a puppet change (the service auto-restarts) [analytics]
11:22 <elukey> disabled temporarily druid's middlemanager on druid1001 to test the Real Time monitor setting [analytics]
2017-12-05 §
10:35 <elukey> re-enable webrequest bundle and camus after reboots [analytics]
10:31 <elukey> disabled druid middlemanager on druid1003 with curl -X POST http://druid1003.eqiad.wmnet:8091/druid/worker/v1/disable [analytics]
10:03 <elukey> stop camus as precautionary measure before Hadoop masters reboot [analytics]
09:57 <elukey> suspend webrequest load bundle as extra precaution before Hadoop masters reboot [analytics]
2017-12-04 §
16:29 <elukey> restart webrequest-load-wf-upload-2017-12-4-12 (failed due to hadoop reboots) [analytics]
16:12 <elukey> restart webrequest-load-wf-upload-2017-12-4-13 (failed due to hadoop reboots) [analytics]
15:09 <joal> Rerun webrequest-load-wf-upload-2017-12-4-12 and webrequest-load-wf-upload-2017-12-4-13 [analytics]
15:08 <joal> Rerunning 15:47:35 < fdans> whatuuuup mforns [analytics]