751-800 of 1412 results (13ms)
2017-01-24 §
18:00 <andrewbogott> apt-get autoremove on tools-mail [tools]
17:51 <andrewbogott> rebooting tools-mail post upgrade [tools]
17:19 <andrewbogott> restarting tools-mail, beginning do-release-upgrade -d -q [tools]
17:17 <andrewbogott> backing up tools-mail to ~root/8c499e6e-1b79-4bb1-8f7f-72fee1f74ea5-backup on labvirt1009 [tools]
17:15 <andrewbogott> stopping tools-mail, backing up, upgrading from precise to trusty [tools]
15:49 <yuvipanda> clush -g all 'sudo rm /usr/local/bin/kube*' to get rid of old kube related binaries [tools]
14:42 <yuvipanda> re-enable puppet on tools-proxy-01, test success on proxy-02 [tools]
14:37 <yuvipanda> disable puppet on tools-proxy-01 (active proxy) to check deploying debianized kube-proxy on proxy-02 [tools]
13:52 <yuvipanda> upgrading k8s on worker nodes to use debs + new k8s version [tools]
13:52 <yuvipanda> finished upgrading k8s + using debs [tools]
12:49 <yuvipanda> purge ancient kubectl, kube-apiserver, kube-controller-manager, kube-scheduler packages from tools-k8s-master-01, these were my old terrible packages [tools]
2017-01-23 §
19:36 <andrewbogott> temporarily shutting down tools-webgrid-lighttpd-1201 [tools]
19:35 <yuvipanda> depool tools-webgrid-lighttpd-1201 for snapshotting tests [tools]
17:13 <chasemp> reboot tools-exec-1411 as having serious transient issues [tools]
2017-01-20 §
15:58 <yuvipanda> enabling puppet across all hosts [tools]
15:36 <yuvipanda> disable puppet everywhere to cherrypick patch moving base to a profile [tools]
00:50 <bd808> sudo qdel -f 1199218 to force delete a stuck toolschecker job [tools]
2017-01-17 §
18:47 <madhuvishy> Reenabled puppet across tools [tools]
18:26 <madhuvishy> Disabling puppet across tools to test https://gerrit.wikimedia.org/r/#/c/329707/ [tools]
2017-01-11 §
22:09 <chasemp> add Reedy to admin in tool labs (approved by bryon and chase for access to investigate specific tool abuse behavior) [tools]
2017-01-10 §
19:05 <madhuvishy> Killed 3 jobs from tools.arnaub that were causing high load on tools-exec-1411 [tools]
2017-01-06 §
19:02 <bd808> Terminated deprecated instances tools-exec-121[2-6] (T154539) [tools]
2017-01-04 §
02:43 <madhuvishy> Reenabled puppet on toolschecker and removed iptables rule on labservices1001 blocking incoming connections from tools-checker-01. T152369 [tools]
2017-01-03 §
23:56 <bd808> Removed tools-exec-12[12-16] from gridengine (T154539) [tools]
23:27 <bd808> drained tools-exec-1216 (T154539) [tools]
23:26 <bd808> drained tools-exec-1215 (T154539) [tools]
23:25 <bd808> drained tools-exec-1214 (T154539) [tools]
23:25 <bd808> drained tools-exec-1213 (T154539) [tools]
23:24 <bd808> drained tools-exec-1212 (T154539) [tools]
23:11 <madhuvishy> Disabled puppet on tools-checker-01 (T152369) [tools]
21:43 <madhuvishy> Adding iptables rule to drop incoming connections from toolschecker on labservices1001 [tools]
20:51 <madhuvishy> Adding iptables rule to block outgoing connections to labservices1001 on tools-checker-01 [tools]
20:43 <madhuvishy> Silenced tools checker on icinga to test labservices1001 failure causing toolschecker to flake out T152369 [tools]
2016-12-25 §
00:28 <yuvipanda> comment out cron running 'clean' script of avicbot every minute without -once [tools]
00:28 <yuvipanda> force delete all jobs of avicbot [tools]
00:25 <yuvipanda> delete all jobs of avicbot. This is 419 jobs [tools]
00:20 <yuvipanda> kill clean.sh process of avicbot [tools]
2016-12-19 §
20:07 <valhallasw`cloud> killed gps_exif_bot2.py (tools.gpsexif), was using 50MB/s io, lagging all of tools-bastion-03 [tools]
13:06 <yuvipanda> run /usr/local/bin/deploy-master http://tools-docker-builder-03.tools.eqiad.wmflabs v1.3.3wmf1 on tools-k8s-master-01 [tools]
12:53 <yuvipanda> cleaned out pbuilder from tools-docker-builder-01 to clean up [tools]
2016-12-17 §
04:49 <yuvipanda> turned on lookupcache again for bastions [tools]
2016-12-15 §
18:52 <yuvipanda> reboot tools-exec-1204 [tools]
18:49 <yuvipanda> reboot tools-webgrid-lighttpd-12[01-05] [tools]
18:45 <yuvipanda> reboot tools-exec-gift [tools]
18:41 <yuvipanda> reboot tools-exec-1217 to 1221 [tools]
18:30 <yuvipanda> rebooted tools-exec-1212 to 1216 [tools]
14:55 <yuvipanda> reboot tools-services-01 [tools]
2016-12-14 §
18:43 <mutante> tools-bastion-03 - ran 'locale-gen ko_KR.EUC-KR' for T130532 [tools]
2016-12-13 §
20:54 <chasemp> reboot bastion-03 as unresponsive [tools]
2016-12-09 §
19:32 <godog> upgrade / restart prometheus-node-exporter [tools]