1001-1050 of 1498 results (14ms)
2016-10-05 §
12:15 <chasemp> reboot tools-webgrid-generic-1404 as locked up [tools]
2016-10-01 §
10:03 <yuvipanda> re-enable puppet on tools-checker-02 [tools]
2016-09-29 §
18:10 <bd808> Investigating elasticsearch cluster issues effecting stashbot [tools]
2016-09-27 §
08:07 <chasemp> tools-bastion-03:~# chmod 640 /var/log/syslog [tools]
2016-09-25 §
15:27 <Krenair> restarted labs-logbot under tools.morebots [tools]
2016-09-21 §
18:56 <madhuvishy> Repooled tools-webgrid-lighttpd-1418 (T146212) after dns records cleanup [tools]
18:42 <madhuvishy> Repooled tools-webgrid-lighttpd-1416 (T146212) after dns records cleanup [tools]
16:57 <chasemp> reboot tools-webgrid-lighttpd-1407, tools-webgrid-lighttpd-1210, tools-webgrid-lighttpd-1414, and then tools-webgrid-lighttpd-1405 as the first 3 return [tools]
2016-09-20 §
23:24 <yuvipanda> depool tools-webgrid-lighttpd-1416 and 1418, they aren't in actual working order [tools]
21:23 <madhuvishy|food> Pooled new sge exec node tools-webgrid-lighttpd-1416 (T146212) [tools]
21:17 <madhuvishy|food> Pooled new sge exec node tools-webgrid-lighttpd-1415 (T146212) [tools]
20:34 <madhuvishy> Created new instance tools-webgrid-lighttpd-1418 (T146212) [tools]
20:34 <madhuvishy> Created new instance tools-webgrid-lighttpd-1416 (T146212) [tools]
20:34 <madhuvishy> Created new instance tools-webgrid-lighttpd-1415 (T146212) [tools]
17:58 <andrewbogott> reboot tools-exec-1410 [tools]
17:54 <yuvipanda> repool tools-webgrid-lighttpd-1412 [tools]
17:49 <yuvipanda> webgrid-lighttpd-1412 hung on io (no change in nova diagnostics), rebooting [tools]
17:33 <yuvipanda> reboot tools-puppetmaster-01 [tools]
17:20 <yuvipanda> reboot tools-checker-02 [tools]
15:42 <chasemp> move floating ip from tools-checker-02 (failed) to tools-checker-01 [tools]
2016-09-13 §
21:09 <madhuvishy> Bumped proxy nginx worker_connections limit T143637 [tools]
21:07 <madhuvishy> Reenabled puppet across proxy hosts [tools]
20:44 <madhuvishy> Disabling puppet across proxy hosts [tools]
2016-09-12 §
18:33 <bd808> Forcing puppet run on tools-cron-01 [tools]
18:31 <bd808> Forcing puppet run on tools-bastion-03 [tools]
18:28 <bd808> Forcing puppet run on tools-bastion-02 [tools]
18:26 <bd808> Forcing puppet run on tools-precise-dev [tools]
18:26 <bd808> Built toollabs-webservice v0.27 package and added to aptly [tools]
2016-09-10 §
01:06 <yuvipanda> migrate tools-k8s-etcd-01 to labvirt1012, is in state doing no io [tools]
2016-09-09 §
19:27 <yuvipanda> reboot tools-exec-1218 and 1219 [tools]
18:10 <yuvipanda> killed massive grep running as root [tools]
2016-09-08 §
21:49 <bd808> forcing puppet runs to install toollabs-webservice_0.26_all.deb [tools]
20:51 <bd808> forcing puppet runs to install jobutils_1.15_all.deb [tools]
2016-09-07 §
21:11 <Krenair> brought labs/private.git up to date on tools-puppetmaster-01 [tools]
02:32 <Krenair> ran `SULWatcher/restart_SULWatcher.sh` as `tools.stewardbots` on bastion-03 to fix T144887 [tools]
2016-09-06 §
22:14 <yuvipanda> got pbuilder off tools-services-01, was taking up too much space. [tools]
22:10 <madhuvishy> Deleted instance tools-web-static-01 and tools-web-static-02 (T143637) [tools]
21:45 <yuvipanda> reboot tools-prometheus-02. nova diagnostics shows no vda activity. [tools]
20:43 <chasemp> drain and reboot tools-exec-1410 for testing [tools]
07:32 <yuvipanda> depooled tools-exec-1219 and 1218, seem to be unresponsive, causing jobs that appear to run but aren't really [tools]
2016-09-05 §
16:27 <andrewbogott> rebooting tools-cron-01 because it is hanging all over the place [tools]
2016-09-01 §
05:19 <yuvipanda> restart maintain-kubeusers on tools-k8s-master-01, was stuck [tools]
2016-08-31 §
20:48 <madhuvishy> Reenabled puppet across tools hosts [tools]
20:45 <madhuvishy> Scratch migration complete on all grid exec nodes (T134896) [tools]
19:36 <madhuvishy> Scratch migration on all non exec/worker nodes complete (T134896) [tools]
18:18 <madhuvishy> Scratch migration complete for all k8s workers (T134896) [tools]
17:50 <madhuvishy> Reenabling puppet across tools hosts. [tools]
16:54 <madhuvishy> Rsync-ed over latest backup of /srv/scratch from labstore1001 to labstore1003 [tools]
2016-08-30 §
18:54 <valhallasw`cloud> edited /etc/shadow on a range of hosts to fix https://phabricator.wikimedia.org/T143191 [tools]
2016-08-29 §
23:38 <Krenair> added myself to the tools.admin service group earlier to try to figure out what was causing the outage, removed again now [tools]