801-850 of 2634 results (11ms)
2019-04-13 §
19:51 <bstorm_> started migrating tools-flannel-etcd-02 to cloudvirt1013 T220853 [tools]
2019-04-11 §
22:38 <andrewbogott> moving tools-paws-worker-1005 to cloudvirt1009 to make spreadcheck happier [tools]
21:49 <bd808> Re-enabled puppet on tools-elastic-02 and forced puppet run [tools]
21:44 <andrewbogott> moving tools-mail-02 to eqiad1-r [tools]
20:56 <andrewbogott> shutting down tools-logs-02 — seems unused [tools]
12:01 <arturo> T151704 deploying oidentd [tools]
11:54 <arturo> disable puppet in all hosts to deploy oidentd [tools]
02:33 <andrewbogott> tools-paws-worker-1005, tools-paws-worker-1006 to eqiad1-r [tools]
00:03 <andrewbogott> tools-paws-worker-1002, tools-paws-worker-1003 to eqiad1-r [tools]
2019-04-10 §
23:58 <andrewbogott> moving tools-clushmaster-02, tools-elastic-03 and tools-paws-worker-1001 to eqiad1-r [tools]
18:52 <bstorm_> depooled and rebooted tools-sgeexec-0929 because systemd was in a weird state [tools]
18:46 <bstorm_> depooled and rebooted tools-sgewebgrid-lighttpd-0913 because high load was caused by ancient lsof processes [tools]
14:49 <bstorm_> cleared E state from 5 queues [tools]
13:06 <arturo> T218126 hard reboot tools-sgeexec-0906 [tools]
12:31 <arturo> T218126 hard reboot tools-sgeexec-0926 [tools]
12:27 <arturo> T218126 hard reboot tools-sgeexec-0925 [tools]
12:06 <arturo> T218126 hard reboot tools-sgeexec-0901 [tools]
11:55 <arturo> T218126 hard reboot tools-sgeexec-0924 [tools]
11:47 <arturo> T218126 hard reboot tools-sgeexec-0921 [tools]
11:23 <arturo> T218126 hard reboot tools-sgeexec-0940 [tools]
11:03 <arturo> T218126 hard reboot tools-sgeexec-0928 [tools]
10:49 <arturo> T218126 hard reboot tools-sgeexec-0923 [tools]
10:43 <arturo> T218126 hard reboot tools-sgeexec-0915 [tools]
10:27 <arturo> T218126 hard reboot tools-sgeexec-0935 [tools]
10:19 <arturo> T218126 hard reboot tools-sgeexec-0914 [tools]
10:01 <arturo> T218126 hard reboot tools-sgeexec-0907 [tools]
09:40 <arturo> T218126 hard reboot tools-sgeexec-0918 [tools]
09:27 <arturo> T218126 hard reboot tools-sgeexec-0932 [tools]
09:26 <arturo> T218216 hard reboot tools-sgeexec-0932 [tools]
09:04 <arturo> T218216 add `profile::ldap::client::labs::client_stack: sssd` to prefix puppet for sge-exec nodes [tools]
09:03 <arturo> T218216 do a controlled rollover of sssd, depooling sgeexec nodes, reboot and repool [tools]
08:39 <arturo> T218216 disable puppet in all tools-sgeexec-XXXX nodes for controlled sssd rollout [tools]
00:32 <andrewbogott> migrating tools-worker-1022, 1023, 1025, 1026 to eqiad1-r [tools]
2019-04-09 §
22:04 <bstorm_> added the new region on port 80 to the elasticsearch security group for stashbot [tools]
20:43 <andrewbogott> moving tools-worker-1018, 1019, 1020, 1021 to eqiad1-r [tools]
20:04 <andrewbogott> moving tools-k8s-etcd-03 to eqiad1-r [tools]
19:54 <andrewbogott> moving tools-flannel-etcd-02 to eqiad1-r [tools]
18:36 <andrewbogott> moving tools-worker-1016, tools-worker-1017 to eqiad1-r [tools]
18:05 <andrewbogott> migrating tools-k8s-etcd-02 to eqiad1-r [tools]
18:00 <andrewbogott> migrating tools-flannel-etcd-01 to eqiad1-r [tools]
17:36 <andrewbogott> moving tools-worker-1014, tools-worker-1015 to eqiad1-r [tools]
17:05 <andrewbogott> migrating tools-k8s-etcd-01 to eqiad1-r [tools]
15:56 <andrewbogott> moving tools-worker-1012, tools-worker-1013 to eqiad1-r [tools]
14:56 <bstorm_> cleared 4 queues on gridengine of E status (ldap again) [tools]
14:07 <andrewbogott> moving tools-worker-1010, tools-worker-1011, tools-worker-1001 to eqiad1-r [tools]
03:48 <andrewbogott> moving tools-worker-1008 and tools-worker-1009 to eqiad1-r [tools]
02:07 <bstorm_> reloaded ferm on tools-flannel-etcd-0[1-3] to get the k8s node moves to register [tools]
2019-04-08 §
22:36 <andrewbogott> moving tools-worker-1006 and tools-worker-1007 to eqiad1-r [tools]
20:03 <andrewbogott> moving tools-worker-1003 and tools-worker-1004 to eqiad1-r [tools]
2019-04-07 §
16:54 <zhuyifei1999_> tools-sgeexec-0928 unresponsive since around 22 UTC. No data on Graphite. Can't ssh in even as root. Hard rebooting via Horizon [tools]