2451-2500 of 3458 results (11ms)
2017-12-19 §
16:34 <elukey> manually started eventlogging cleaner on db1107 to purge/sanitize data up to 90 days ago (tmux is running for user eventlogcleaner) [analytics]
14:10 <elukey> temporary changed JVM Heap settings for the druid broker on druid1001 - Xmx25g Xms10g (run puppet and restart the daemon to rollback) [analytics]
2017-12-12 §
15:54 <milimetric> sieging aqs1004 with 100.000 transactions [analytics]
2017-12-11 §
21:16 <andrewbogott> updating puppetmaster p1.analytics.eqiad.wmflabs to puppet 4 [analytics]
14:07 <elukey> disable druid middlemanager on druid1003 to drain + restart to pick up new logging settings [analytics]
12:59 <elukey> disabled druid middlemanager on druid1002 to drain+restart with new logging config [analytics]
2017-12-08 §
11:15 <joal> Start mediawiki-history oozie jobs new-version [analytics]
10:49 <joal> Update wmf.mediawiki_history as explained in email (rename current table to old, create new one) [analytics]
2017-12-07 §
21:09 <joal> Start clickstream oozie job [analytics]
20:45 <joal> Kill restbase oozie job and restart apis replacing one [analytics]
20:12 <joal> Trying to deploy refinery again [analytics]
19:30 <joal> Deploying refinery now that -source is deployed [analytics]
18:39 <milimetric> Deployed refinery-source using jenkins [analytics]
15:03 <elukey> restart webrequest-misc load job (Dec 7 2017 06:00:00) [analytics]
12:24 <elukey> camus re-enabled after analytics1003 reboot [analytics]
08:31 <elukey> stop camus on an1003 as prep step for reboot [analytics]
2017-12-06 §
14:55 <elukey> restart hue to pick up the new oozie server [analytics]
14:55 <elukey> oozie server accidentally restarted due to a puppet change (the service auto-restarts) [analytics]
11:22 <elukey> disabled temporarily druid's middlemanager on druid1001 to test the Real Time monitor setting [analytics]
2017-12-05 §
10:35 <elukey> re-enable webrequest bundle and camus after reboots [analytics]
10:31 <elukey> disabled druid middlemanager on druid1003 with curl -X POST http://druid1003.eqiad.wmnet:8091/druid/worker/v1/disable [analytics]
10:03 <elukey> stop camus as precautionary measure before Hadoop masters reboot [analytics]
09:57 <elukey> suspend webrequest load bundle as extra precaution before Hadoop masters reboot [analytics]
2017-12-04 §
16:29 <elukey> restart webrequest-load-wf-upload-2017-12-4-12 (failed due to hadoop reboots) [analytics]
16:12 <elukey> restart webrequest-load-wf-upload-2017-12-4-13 (failed due to hadoop reboots) [analytics]
15:09 <joal> Rerun webrequest-load-wf-upload-2017-12-4-12 and webrequest-load-wf-upload-2017-12-4-13 [analytics]
15:08 <joal> Rerunning 15:47:35 < fdans> whatuuuup mforns [analytics]
14:17 <elukey> re-run pageview-druid-hourly-wf-2017-12-4-11 in Hue (failed due to reboots) [analytics]
12:04 <elukey> re-run webrequest-load-wf-upload-2017-12-4-8 (failed due to reboots) [analytics]
12:04 <elukey> re-run webrequest-load-check_sequence_statistics-wf-upload-2017-12-4-7 (failed due to reboots) [analytics]
2017-12-02 §
11:47 <joal> Rerun unique_devices-per_project_family-monthly-wf-2017-11 [analytics]
2017-12-01 §
15:20 <elukey> rerun webrequest-druid-hourly-wf-2017-12-1-8 after an unexpected Druid Overlord inconsistency [analytics]
15:09 <elukey> rerun pageview-druid-hourly-wf-2017-12-1-8 after an unexpected Druid Overlord inconsistency [analytics]
13:07 <elukey> re-run aqs-hourly-wf-2017-12-1-8 (failed due to Hadoop reboots) [analytics]
12:42 <elukey> temporarily switch pivot's config to druid1002 (to reboot druid1001) [analytics]
12:37 <elukey> re-run webrequest-load-wf-upload-2017-12-1-10 and webrequest-load-wf-upload-2017-12-1-7 (failed due to Hadoop reboots) [analytics]
12:36 <elukey> re-run webrequest-load-wf-text-2017-12-1-10 and webrequest-load-wf-text-2017-12-1-9 (failed due to Hadoop reboots) [analytics]
12:35 <elukey> re-run pageview-hourly-wf-2017-12-1-8 (failed due to Hadoop reboots) [analytics]
12:34 <elukey> re-run webrequest-druid-hourly-wf-2017-12-1-8 (failed due to Hadoop reboots) [analytics]
2017-11-30 §
18:20 <elukey> re-run webrequest-load-wf-upload-2017-11-30-16 (failed due to hadoop reboots) [analytics]
18:19 <elukey> re-run webrequest-load-wf-text-2017-11-30-14 (failed due to hadoop reboots) [analytics]
16:21 <joal> wikidata-wdqs_extract-wf-2017-11-30-15 [analytics]
15:50 <elukey> restart hue on thorium - timeouts and 500s [analytics]
14:58 <joal> Update druid overlord config to equalDistribution dynamically [analytics]
2017-11-29 §
21:46 <joal> rerun pageview-druid-hourly-wf-2017-11-29-18 and pageview-druid-hourly-wf-2017-11-29-19 [analytics]
21:19 <joal> rerun webrequest-druid-hourly-wf-2017-11-29-18 [analytics]
2017-11-28 §
14:41 <ottomata> restarting eventlogging on eventlog1001 for https://gerrit.wikimedia.org/r/#/c/393613/ [analytics]
09:08 <elukey> log database on dbstore1002 dropped for good [analytics]
2017-11-22 §
16:09 <ottomata> restarting eventlogging services on eventlog1001 [analytics]
2017-11-20 §
18:28 <elukey> deployed prometheus-druid-exporter (still not released in apt) on druid1004 for testing [analytics]