801-850 of 2884 results (15ms)
2019-10-07 §
18:32 <phamhi> reboot tools-worker-1015 due to nfs stale issue [tools]
18:32 <phamhi> reboot tools-worker-1014 due to nfs stale issue [tools]
18:23 <phamhi> reboot tools-worker-1013 due to nfs stale issue [tools]
18:21 <phamhi> reboot tools-worker-1012 due to nfs stale issue [tools]
18:12 <phamhi> reboot tools-worker-1011 due to nfs stale issue [tools]
18:12 <phamhi> reboot tools-worker-1010 due to nfs stale issue [tools]
18:08 <phamhi> reboot tools-worker-1009 due to nfs stale issue [tools]
18:07 <phamhi> reboot tools-worker-1008 due to nfs stale issue [tools]
17:58 <phamhi> reboot tools-worker-1007 due to nfs stale issue [tools]
17:57 <phamhi> reboot tools-worker-1006 due to nfs stale issue [tools]
17:47 <phamhi> reboot tools-worker-1005 due to nfs stale issue [tools]
17:47 <phamhi> reboot tools-worker-1004 due to nfs stale issue [tools]
17:43 <phamhi> reboot tools-worker-1002.tools.eqiad.wmflabs due to nfs stale issue [tools]
17:35 <phamhi> drained and uncordoned tools-worker-100[1-5] [tools]
17:32 <bstorm_> reboot tools-sgewebgrid-lighttpd-0912 [tools]
17:30 <bstorm_> reboot tools-sgewebgrid-lighttpd-0923/24/08 [tools]
17:01 <bstorm_> rebooting tools-sgegrid-master and tools-sgegrid-shadow 😭 [tools]
16:58 <bstorm_> rebooting tools-sgewebgrid-lighttpd-0902/4/6/7/8/19 [tools]
16:53 <bstorm_> rebooting tools-sgewebgrid-generic-0902/4 [tools]
16:50 <bstorm_> rebooting tools-sgeexec-0915/18/19/23/26 [tools]
16:49 <bstorm_> rebooting tools-sgeexec-0901 and tools-sgeexec-0909/10/11 [tools]
16:46 <bd808> `sudo shutdown -r now` for tools-sgebastion-08 [tools]
16:41 <bstorm_> reboot tools-sgebastion-07 [tools]
16:39 <bd808> `sudo service nslcd restart` on tools-sgebastion-08 [tools]
2019-10-04 §
21:43 <bd808> `sudo exec-manage repool tools-sgeexec-0923.tools.eqiad.wmflabs` [tools]
21:26 <bd808> Rebooting tools-sgeexec-0923 after lots of messing about with a broken update-initramfs build [tools]
20:35 <bd808> Manually running `/usr/bin/python3 /usr/bin/unattended-upgrade` on tools-sgeexec-0923 [tools]
20:33 <bd808> Killed 2 /usr/bin/unattended-upgrade procs on tools-sgeexec-0923 that seemed stuck [tools]
13:33 <arturo> remove /etc/init.d/rsyslog on tools-worker-XXXX nodes so the rsyslog deb prerm script doesn't prevent the package from being updated [tools]
2019-10-03 §
13:05 <arturo> delete servers tools-sssd-sgeexec-test-[1,2], no longer required [tools]
2019-09-27 §
16:59 <bd808> Set "profile::rsyslog::kafka_shipper::kafka_brokers: []" in tools-elastic prefix puppet [tools]
00:40 <bstorm_> depooled and rebooted tools-sgewebgrid-lighttpd-0927 [tools]
2019-09-25 §
19:08 <andrewbogott> moving tools-sgewebgrid-lighttpd-0903 to cloudvirt1021 [tools]
2019-09-23 §
16:58 <bstorm_> deployed tools-manifest 0.20 and restarted webservicemonitor [tools]
06:01 <bd808> Restarted maintain-dbusers process on labstore1004. (T233530) [tools]
2019-09-12 §
20:48 <phamhi> Deleted tools-puppetdb-01.tools as it is no longer in used [tools]
2019-09-11 §
13:30 <jeh> restart tools-sgeexec-0912 [tools]
2019-09-09 §
22:44 <bstorm_> uncordoned tools-worker-1030 and tools-worker-1038 [tools]
2019-09-06 §
15:11 <bd808> `sudo kill -9 10635` on tools-k8s-master-01 (T194859) [tools]
2019-09-05 §
21:02 <bd808> Enabled Puppet on tools-docker-registry-03 and forced puppet run (T232135) [tools]
18:13 <bd808> Disabled Puppet on tools-docker-registry-03 to investigate docker-registry issue (no phab task yet) [tools]
2019-09-01 §
20:51 <Reedy> `sudo service maintain-kubeusers restart` on tools-k8s-master-01 [tools]
2019-08-30 §
16:54 <phamhi> restart maintain-kuberusers service in tools-k8s-master-01 [tools]
16:21 <bstorm_> depooling tools-sgewebgrid-lighttpd-0923 to reboot -- high iowait likely from NFS mounts [tools]
2019-08-29 §
22:18 <bd808> Finished building new stretch Docker images for Toolforge Kubernetes use [tools]
22:06 <bd808> Starting process of building new stretch Docker images for Toolforge Kubernetes use [tools]
22:05 <bd808> Jessie Docker image rebuild complete [tools]
21:31 <bd808> Starting process of building new jessie Docker images for Toolforge Kubernetes use [tools]
2019-08-27 §
19:10 <bd808> Restarted maintain-kubeusers after complaint on irc. It was stuck in limbo again [tools]
2019-08-26 §
21:48 <bstorm_> repooled tools-sgewebgrid-generic-0902, tools-sgewebgrid-lighttpd-0902, tools-sgewebgrid-lighttpd-0903 and tools-sgeexec-0905 [tools]