8951-9000 of 10000 results (31ms)
2018-03-16 §
08:40 <elukey> reboot druid1006 for kernel updates [production]
08:29 <elukey> reboot druid1005 for kernel updates [production]
2018-03-15 §
15:20 <elukey> reboot druid1003 for kernel updates [production]
14:30 <elukey> reboot druid1004 for kernel updates [production]
13:51 <elukey> reboot kafka1001 (eventbus/job-queues eqiad) for kernel updates [production]
2018-03-12 §
09:56 <elukey> restart kafka mirror maker (main eqiad -> jumbo) on kafka1020 (all consumers not assigned to any partition on kafka102*) [production]
2018-03-11 §
08:50 <elukey> executed sudo rm /etc/logrotate.d/kafkatee-webrequest-analytics on oxygen/rhenium to stop daily cronspam [production]
2018-03-09 §
12:41 <elukey> manually executed systemctl reset-failed to some old (not present anymore) units on kafka analytics hosts [production]
2018-03-08 §
13:40 <elukey> eventlogging analytics migrated from eventlog1001 to eventlog1002 [production]
08:58 <elukey> restart varnish backend on cp3041 (failed fetches) [production]
08:50 <elukey> rebooting analytics1003 (Hadoop Hive, Oozie, etc..) for kernel updates [production]
08:31 <elukey> reboot analytics1002 (Hadoop master standby) for kernel upgrades [production]
08:19 <elukey> reboot analytics1001 (Hadoop master) for kernel upgrade (temp failover to analytics1002) [production]
07:44 <elukey> reboot kafka2003 (eventbus codfw) for kernel updates [production]
07:24 <elukey> reboot kafka2002 (eventbus codfw) for kernel updates [production]
2018-03-07 §
16:08 <elukey> updating pcc facts for new hosts [production]
10:50 <elukey> reboot stat100[56] for kernel upgrades [production]
10:03 <elukey> reboot analytics10[35,52] for kernel updates - hadoop hdfs journal nodes (didn't manage to complete the work yesterday) [production]
2018-03-06 §
11:05 <elukey> reboot analytics10[28,35,52] for kernel updates (one at the time, hadoop hdfs journal nodes) [production]
09:27 <elukey> reboot kafka2001 (eventbus codfw) for kernel updates [production]
08:50 <elukey> reboot meitnerium (archiva) for kernel updates [production]
08:30 <elukey> drain+reboot analytics[1065-1067] for kernel updates [production]
08:01 <elukey> drain+reboot analytics[61,63,64] for kernel updates [production]
2018-03-05 §
17:34 <elukey> drain + reboot analytics10[58-60] for kernel updates [production]
16:00 <elukey> test [production]
15:41 <elukey> drain + reboot analytics 1055->57 for kernel updates [production]
14:34 <elukey> graphite metrics mw.error.* deprecated in T188749 [production]
11:09 <elukey> drain + reboot analytics10[50,51,53,54] for kernel updates [production]
10:24 <elukey> drain + reboot analytics10[46-49] for kernel updates [production]
2018-03-04 §
15:59 <elukey> powercycle stat1004 - available via mgmt, root login freezes while trying [production]
2018-03-02 §
15:19 <elukey> drain + reboot analytics10[41-45] for kernel updates [production]
13:46 <elukey> drain + reboot analytics10[38,39,40,41] for kernel updates [production]
13:22 <elukey> drain + reboot analytics10[33,34,36,37] for kernel updates [production]
11:58 <elukey> drain + reboot analytics10[29,31,32] for kernel updates [production]
10:01 <elukey> deleted /etc/burrow/* from zookeeper main eqiad/codfw after https://gerrit.wikimedia.org/r/415818 (garbage to cleanup) [production]
2018-03-01 §
13:17 <elukey> reboot kafka-jumbo100[5,6] for kernel updates [production]
12:27 <elukey> reboot kafka-jumbo1004 for kernel updates [production]
12:21 <elukey> reboot kafka1023 for kernel updates [production]
11:36 <elukey> reboot kafka-jumbo1003 for kernel updates [production]
11:32 <elukey> reboot kafka1022 for kernel updates [production]
11:20 <elukey> reboot kafka-jumbo1002 for kernel security updates [production]
11:08 <elukey> reboot kafka1020 for kernel updates [production]
09:59 <elukey> reboot kafka1014 for kernel security updates [production]
09:43 <elukey> reboot kafka1013 for kernel security updates [production]
09:29 <elukey> rebooting analytics1030 for kernel updates [production]
08:34 <elukey> reboot kafka1012 for kernel updates - T188594 [production]
07:55 <elukey> reboot kafka-jumbo1001 for kerne updates - T188594 [production]
07:52 <elukey> run kafka preferred-replica-election on kafka1012 to force broker 18 to get back among Kafka topic leaders [production]
2018-02-27 §
16:53 <elukey> restart cassandra-a on aqs1004 to test the prometheus jmx agent before complete rollout - T184795 [production]
2018-02-26 §
09:23 <elukey> copied burrow 0.1 from jessie-wikimedia to stretch-wikimedia [production]