551-600 of 1036 results (11ms)
2016-11-01 §
17:52 <joal> Deploying refinery [analytics]
14:45 <joal> Restart webrequest load job to apply [analytics]
14:33 <joal> deploying refinery onto the cluster [analytics]
14:00 <ottomata> restarting pivot [analytics]
2016-10-31 §
17:09 <ottomata> bouncing eventlogging [analytics]
17:00 <ottomata> kafka preferred replica election on main-eqiad kafka cluster to promote kafka1003 as leader for its preferred partitions [analytics]
14:49 <ottomata> adding kafka1003 in as replicas for active main-eqiad topics [analytics]
14:12 <ottomata> adding kafka1003 as kafka broker in main-eqiad cluster [analytics]
14:00 <joal> deploy refinery [analytics]
2016-10-28 §
13:04 <elukey> oozie firewall rules changed - nowonly the analytics network is allowed [analytics]
00:19 <bd808> Testing logging to mw.o SAL via stashbot [analytics]
2016-10-24 §
15:29 <mforns> actually it was webrequest-load-check_sequence_statistics-wf-upload-2016-10-24-14 [analytics]
15:29 <mforns> created 0005238-161020124223818-oozie-oozi-C to re-run webrequest-load-check_sequence_statistics-wf-upload-2016-10-21-17 (oozie errors) [analytics]
2016-10-22 §
13:45 <mforns> created 0002687-161020124223818-oozie-oozi-C to re-run webrequest-load-check_sequence_statistics-wf-upload-2016-10-21-17 (oozie errors) [analytics]
12:44 <elukey> created 0002631-161020124223818-oozie-oozi-C to re-run webrequest-load-check_sequence_statistics-wf-upload-2016-10-21-17 (oozie errors) [analytics]
2016-10-21 §
15:33 <elukey> created 0001564-161020124223818-oozie-oozi-C to re-run webrequest-load-check_sequence_statistics-wf-upload-2016-10-21-14 (oozie errors) [analytics]
12:29 <elukey> created 0001387-161020124223818-oozie-oozi-C for webrequest-load-check_sequence_statistics-wf-upload-2016-10-21-11 (oozie errors) [analytics]
10:33 <elukey> re-run webrequest-load-wf-text-2016-10-21-00 and webrequest-load-wf-maps-2016-10-21-00 [analytics]
05:52 <elukey> created 0001009-161020124223818-oozie-oozi-C to run webrequest-load-check_sequence_statistics-wf-upload-2016-10-21-3 (oozie errors) [analytics]
2016-10-20 §
18:38 <mforns> created 0000390-161020124223818-oozie-oozi-C to re-run webrequest-load-check_sequence_statistics-wf-upload-2016-10-20-14&15 (oozie errors) [analytics]
17:06 <elukey> created 0000294-161020124223818-oozie-oozi-C to re-run webrequest-load-check_sequence_statistics-wf-upload-2016-10-20-13 (oozie errors) [analytics]
16:17 <ottomata> restarting eventlogging after rebooting kafka brokers [analytics]
15:43 <mforns> restarted EventLogging after throughput drop [analytics]
14:20 <ottomata> starting rolling restart of analytics-eqiad kafka brokers to apply kernel update [analytics]
13:13 <elukey> re-enabling oozie and camus after cluster reboot [analytics]
10:04 <elukey> stopped camus on an1027 and all the oozie bundles as prep step for the reboots of analytics* [analytics]
07:49 <elukey> created 0038058-160922102909979-oozie-oozi-C to re-run webrequest-load-check_sequence_statistics-wf-upload-2016-10-20-5 (oozie errors) [analytics]
07:48 <elukey> created 0038054-160922102909979-oozie-oozi-C to re-run webrequest-load-check_sequence_statistics-wf-upload-2016-10-20-3 (oozie errors) [analytics]
02:14 <andrewbogott> restarted druid101 because it was in some kind of kernel lockup [analytics]
2016-10-18 §
11:55 <elukey> removed /etc/mysql/conf.d/research-client.cnf from stat1002 (root:root perms, not supposed to be there but only on stat1003) [analytics]
09:24 <elukey> merged puppet change to force varnishkafka to use -q 'ReqMethod ne "PURGE" and not Timestamp:Pipe and not ReqHeader:Upgrade ~ "[wW]ebsocket" and not HttpGarbage' and -L 10000 [analytics]
06:16 <elukey> created the oozie coordinator 0035451-160922102909979-oozie-oozi-C for webrequest-load-check_sequence_statistics-wf-upload-2016-10-18-5 [analytics]
05:51 <elukey> created the oozie coordinator 0035415-160922102909979-oozie-oozi-C for webrequest-load-check_sequence_statistics-wf-upload-2016-10-18-[234] [analytics]
2016-10-17 §
16:57 <elukey> started the oozie coordinator 0034720-160922102909979-oozie-oozi-C to re-execute webrequest-load-wf-upload-2016-10-17-14 [analytics]
16:42 <ottomata> restarting hadoop nodemanagers 1 at a time [analytics]
15:32 <ottomata> rebootting analytics1030 [analytics]
08:24 <elukey> upgraded nodejs on aqs100[56] (already done on aqs1004) [analytics]
07:35 <elukey> created oozie coordinator 0034240-160922102909979-oozie-oozi-C to restart webrequest-load-check_sequence_statistics-wf-upload-2016-10-17-6 [analytics]
06:18 <elukey> created oozie coordinator 0034161-160922102909979-oozie-oozi-C to restart webrequest-load-check_sequence_statistics-wf-upload-2016-10-17-4 [analytics]
06:17 <elukey> created oozie coordinator 0034153-160922102909979-oozie-oozi-C to restart webrequest-load-check_sequence_statistics-wf-upload-2016-10-17-3 [analytics]
06:16 <elukey> created oozie coordinator 0034149-160922102909979-oozie-oozi-C to restart webrequest-load-check_sequence_statistics-wf-upload-2016-10-17-2 [analytics]
06:15 <elukey> created oozie coordinator 0034143-160922102909979-oozie-oozi-C to restart webrequest-load-check_sequence_statistics-wf-upload-2016-10-17-1 [analytics]
2016-10-13 §
06:57 <elukey> launched 0029282-160922102909979-oozie-oozi-C to re-run webrequest-load-check_sequence_statistics-wf-upload-2016-10-13-5 with higher error threshold [analytics]
06:56 <elukey> launched 0029278-160922102909979-oozie-oozi-C to re-run webrequest-load-check_sequence_statistics-wf-upload-2016-10-13-2 with higher error threshold [analytics]
2016-10-11 §
18:57 <elukey> kafka1018 back in service after maintenace [analytics]
13:44 <elukey> merged https://gerrit.wikimedia.org/r/315101 on stat1002 (removal of ::statistics:wikistats) [analytics]
13:00 <joal> Start prod version of wdqs_extract job [analytics]
12:16 <joal> Deploying refinery [analytics]
11:17 <elukey> started 0026094-160922102909979-oozie-oozi-C to fix webrequest-load-check_sequence_statistics-wf-upload-2016-10-11-9 (oozie data consistency errors) [analytics]
08:22 <joal> Killing cassandra loading jobs for old aqs [analytics]