2021-02-09
§
|
16:30 |
<elukey> |
restart datanode on ana-worker1100 |
[analytics] |
16:14 |
<ottomata> |
restart datanode on analytics1059 with 16g heap |
[analytics] |
16:08 |
<ottomata> |
restart datanode on an-worker1080 withh 16g heap |
[analytics] |
15:58 |
<ottomata> |
restart datanode on analytics1058 |
[analytics] |
15:55 |
<ottomata> |
restart datenode on an-worker1115 |
[analytics] |
15:38 |
<elukey> |
restart namenode on an-master1002 |
[analytics] |
15:01 |
<elukey> |
restart an-worker1104 with 16g heap size to allow bootstrap |
[analytics] |
15:01 |
<elukey> |
restart an-worker1103 with 16g heap size to allow bootstrap |
[analytics] |
14:57 |
<elukey> |
restart an-worker1102 with 16g heap size to allow bootstrap |
[analytics] |
14:54 |
<elukey> |
restart an-worker1090 with 16g heap size to allow bootstrap |
[analytics] |
14:50 |
<elukey> |
restart analytics1072 with 16g heap size to allow bootstrap |
[analytics] |
14:50 |
<elukey> |
restart analytics1069 with 16g heap size to allow bootstrap |
[analytics] |
14:08 |
<elukey> |
restart analytics1069's datanode with bigger heap size |
[analytics] |
13:39 |
<elukey> |
restart hdfs-datanode on analytics10[65,69] - failed to bootstrap due to issues reading datanode dirs |
[analytics] |
13:38 |
<elukey> |
restart hdfs-datanode on an-worker1080 (test canary - not showing up in block report) |
[analytics] |
10:04 |
<elukey> |
stop mysql replication an-coord1001 -> an-coord1002, an-coord1001 -> db1108 |
[analytics] |
08:29 |
<elukey> |
leave hdfs safemode to let distcp do its job |
[analytics] |
08:25 |
<elukey> |
set hdfs safemode on for the Analytics cluster |
[analytics] |
08:19 |
<elukey> |
umount /mnt/hdfs from all nodes using it |
[analytics] |
08:16 |
<joal> |
Kill flink yarn app |
[analytics] |
08:08 |
<elukey> |
stop jupyterhub on stat100x |
[analytics] |
08:07 |
<elukey> |
stop hive on an-coord100[1,2] - prep step for bigtop upgrade |
[analytics] |
08:05 |
<elukey> |
stop oozie an-coord1001 - prep step for bigtop upgrade |
[analytics] |
08:03 |
<elukey> |
stop presto-server on an-presto100x and an-coord1001 - prep step for bigtop upgrade |
[analytics] |
07:28 |
<elukey> |
roll out new apt bigtop changes across all hadoop-related nodes |
[analytics] |
07:19 |
<joal> |
Killing yarn users applications |
[analytics] |
07:12 |
<elukey> |
stop airflow on an-airflow1001 (prep step for bigtop) |
[analytics] |
07:09 |
<elukey> |
stop namenode on an-worker1124 (backup cluster), create two new partitions for backup and namenode, restart namenode |
[analytics] |
06:14 |
<elukey> |
disable timers on labstore nodes (prep step for bigtop) |
[analytics] |
06:11 |
<elukey> |
disable systemd timers on an-launcher1002 (prep step for bigtop) |
[analytics] |