751-800 of 2823 results (16ms)
2019-10-07 §
17:47 <phamhi> reboot tools-worker-1004 due to nfs stale issue [tools]
17:43 <phamhi> reboot tools-worker-1002.tools.eqiad.wmflabs due to nfs stale issue [tools]
17:35 <phamhi> drained and uncordoned tools-worker-100[1-5] [tools]
17:32 <bstorm_> reboot tools-sgewebgrid-lighttpd-0912 [tools]
17:30 <bstorm_> reboot tools-sgewebgrid-lighttpd-0923/24/08 [tools]
17:01 <bstorm_> rebooting tools-sgegrid-master and tools-sgegrid-shadow 😭 [tools]
16:58 <bstorm_> rebooting tools-sgewebgrid-lighttpd-0902/4/6/7/8/19 [tools]
16:53 <bstorm_> rebooting tools-sgewebgrid-generic-0902/4 [tools]
16:50 <bstorm_> rebooting tools-sgeexec-0915/18/19/23/26 [tools]
16:49 <bstorm_> rebooting tools-sgeexec-0901 and tools-sgeexec-0909/10/11 [tools]
16:46 <bd808> `sudo shutdown -r now` for tools-sgebastion-08 [tools]
16:41 <bstorm_> reboot tools-sgebastion-07 [tools]
16:39 <bd808> `sudo service nslcd restart` on tools-sgebastion-08 [tools]
2019-10-04 §
21:43 <bd808> `sudo exec-manage repool tools-sgeexec-0923.tools.eqiad.wmflabs` [tools]
21:26 <bd808> Rebooting tools-sgeexec-0923 after lots of messing about with a broken update-initramfs build [tools]
20:35 <bd808> Manually running `/usr/bin/python3 /usr/bin/unattended-upgrade` on tools-sgeexec-0923 [tools]
20:33 <bd808> Killed 2 /usr/bin/unattended-upgrade procs on tools-sgeexec-0923 that seemed stuck [tools]
13:33 <arturo> remove /etc/init.d/rsyslog on tools-worker-XXXX nodes so the rsyslog deb prerm script doesn't prevent the package from being updated [tools]
2019-10-03 §
13:05 <arturo> delete servers tools-sssd-sgeexec-test-[1,2], no longer required [tools]
2019-09-27 §
16:59 <bd808> Set "profile::rsyslog::kafka_shipper::kafka_brokers: []" in tools-elastic prefix puppet [tools]
00:40 <bstorm_> depooled and rebooted tools-sgewebgrid-lighttpd-0927 [tools]
2019-09-25 §
19:08 <andrewbogott> moving tools-sgewebgrid-lighttpd-0903 to cloudvirt1021 [tools]
2019-09-23 §
16:58 <bstorm_> deployed tools-manifest 0.20 and restarted webservicemonitor [tools]
06:01 <bd808> Restarted maintain-dbusers process on labstore1004. (T233530) [tools]
2019-09-12 §
20:48 <phamhi> Deleted tools-puppetdb-01.tools as it is no longer in used [tools]
2019-09-11 §
13:30 <jeh> restart tools-sgeexec-0912 [tools]
2019-09-09 §
22:44 <bstorm_> uncordoned tools-worker-1030 and tools-worker-1038 [tools]
2019-09-06 §
15:11 <bd808> `sudo kill -9 10635` on tools-k8s-master-01 (T194859) [tools]
2019-09-05 §
21:02 <bd808> Enabled Puppet on tools-docker-registry-03 and forced puppet run (T232135) [tools]
18:13 <bd808> Disabled Puppet on tools-docker-registry-03 to investigate docker-registry issue (no phab task yet) [tools]
2019-09-01 §
20:51 <Reedy> `sudo service maintain-kubeusers restart` on tools-k8s-master-01 [tools]
2019-08-30 §
16:54 <phamhi> restart maintain-kuberusers service in tools-k8s-master-01 [tools]
16:21 <bstorm_> depooling tools-sgewebgrid-lighttpd-0923 to reboot -- high iowait likely from NFS mounts [tools]
2019-08-29 §
22:18 <bd808> Finished building new stretch Docker images for Toolforge Kubernetes use [tools]
22:06 <bd808> Starting process of building new stretch Docker images for Toolforge Kubernetes use [tools]
22:05 <bd808> Jessie Docker image rebuild complete [tools]
21:31 <bd808> Starting process of building new jessie Docker images for Toolforge Kubernetes use [tools]
2019-08-27 §
19:10 <bd808> Restarted maintain-kubeusers after complaint on irc. It was stuck in limbo again [tools]
2019-08-26 §
21:48 <bstorm_> repooled tools-sgewebgrid-generic-0902, tools-sgewebgrid-lighttpd-0902, tools-sgewebgrid-lighttpd-0903 and tools-sgeexec-0905 [tools]
2019-08-18 §
08:11 <arturo> restart maintain-kuberusers service in tools-k8s-master-01 [tools]
2019-08-17 §
10:56 <arturo> force-reboot tools-worker-1006. Is completely stuck [tools]
2019-08-15 §
15:32 <jeh> upgraded jobutils debian package to 1.38 T229551 [tools]
09:22 <arturo> restart maintain-kubeusers service in tools-k8s-master-01 because some tools were missing their namespaces [tools]
2019-08-13 §
22:00 <bstorm_> truncated exim paniclog on tools-sgecron-01 because it was being spammy [tools]
13:41 <jeh> Set icingia downtime for toolschecker labs showmount T229448 [tools]
2019-08-12 §
16:08 <phamhi> updated prometheus-node-exporter from 0.14.0~git20170523-1 to 0.17.0+ds-3 in tools-worker-[1030-1040] nodes (T230147) [tools]
2019-08-08 §
19:25 <jeh> restarting tools-sgewebgrid-lighttpd-0915 T230157 [tools]
2019-08-07 §
19:07 <bd808> Disassociated SUL and Phabricator accounts from user Lophi (T229713) [tools]
2019-08-06 §
16:18 <arturo> add phamhi as user/projectadmin (T228942) and delete hpham [tools]
15:58 <arturo> add hpham as user/projectadmin (T228942) [tools]