51-100 of 625 results (13ms)
2016-11-30 §
04:52 <bd808> drained tools-exec-1201 (T151980) [tools]
04:48 <bd808> draining tools-exec-1201 [tools]
2016-11-29 §
13:43 <hashar> updating jouncebot so it properly reclaim its nick ( T150916 https://gerrit.wikimedia.org/r/#/c/324025/ ) [tools]
2016-11-22 §
15:13 <chasemp> readd attr +i to replica.my.cnf that seems to have gotten lost in rsync migration [tools]
2016-11-21 §
21:15 <YuviPanda> disable puppet everywhere [tools]
19:49 <YuviPanda> restart all webservice jobs on gridengine to pick up logging again [tools]
2016-11-20 §
06:51 <Krenair> ran `qmod -rj lighttpd-admin` as tools.admin to try to get the main page back up, it worked briefly but then broke again [tools]
2016-11-16 §
20:14 <yuvipanda> upgrade toollabs-webservice to 0.30 on all webgrid nodes [tools]
18:31 <chasemp> reboot tools-exec-1404 (already depooled) [tools]
18:19 <chasemp> reboot tools-exec-1403 [tools]
17:23 <chasemp> reboot tools-exec-1212 (converted via 321786 testing for recovery on boot) [tools]
16:55 <chasemp> clush -g all "puppet agent --disable 'trail run for changeset 321786 handling /var/lib/gridengine'" [tools]
02:05 <yuvipanda> rebooting tools-docker-registry-01, can't ssh in [tools]
01:43 <yuvipanda> cleanup old images on tools-docker-builder-03 [tools]
2016-11-15 §
19:52 <chasemp> reboot tools-precise-dev [tools]
05:20 <yuvipanda> restart all k8s webservices too [tools]
05:05 <yuvipanda> restarting all webservices on gridengine [tools]
03:21 <chasemp> reboot tools-checker-01 [tools]
02:56 <chasemp> reboot tools-exec-1405 to ensure noauto works (because atboot=>false is lies) [tools]
02:31 <chasemp> reboot tools-exec-1406 [tools]
2016-11-14 §
22:51 <chasemp> shut down bastion 02 and 05 and make 03 root only [tools]
19:35 <madhuvishy> Stopped cron on tools-cron-01 (T146154) [tools]
18:24 <madhuvishy> Tools NFS is read-only. /data/project and /home across tools are ro T146154 [tools]
16:57 <yuvipanda> stopped gridengine master [tools]
16:47 <yuvipanda> start restarting kubernetes webservice pods [tools]
16:30 <madhuvishy> Unmounted all nfs shares from tools-k8s-master-01 (sudo /usr/local/sbin/nfs-mount-manager clean) T146154 [tools]
16:22 <yuvipanda> kill maintain-kubeusers on tools-k8s-master-01, sole process touching NFS [tools]
16:22 <chasemp> enable puppet and run on tools-services-01 [tools]
16:21 <yuvipanda> restarting all webservice jobs, watching webservicewatcher logs on tools-services-02 [tools]
16:14 <madhuvishy> Disabling puppet across tools T146154 [tools]
2016-11-11 §
20:49 <madhuvishy> Dual mount of tools share complete. Puppet reenabled across tools hosts. T146154 [tools]
20:18 <madhuvishy> Rolling out dual mount of tools share across all hosts T146154 [tools]
19:29 <madhuvishy> Disabling puppet across tools to dual mount tools share from labstore-secondary T146154 [tools]
2016-11-02 §
18:23 <yuvipanda> manually stop tools-grid-master for reboot [tools]
17:42 <yuvipanda> drain nodes from labvirt1012 and 13 [tools]
13:42 <chasemp> depool tools-exec-1404 for maint [tools]
2016-11-01 §
21:54 <yuvipanda> stop gridengine-master on tools-grid-master in preparation for reboot [tools]
21:34 <yuvipanda> depool tools nodes on labvirt1012 [tools]
21:16 <yuvipanda> depool things in labvirt1011 [tools]
20:58 <yuvipanda> depool tools nodes on labvirt1010 [tools]
20:32 <yuvipanda> depool tools things on labvirt1005 and 1009 [tools]
20:08 <yuvipanda> depooled things on labvirt1006 and 1008 [tools]
19:51 <yuvipanda> move tools-elastic-03 to labvirt1010, -02 already in 09 [tools]
19:34 <yuvipanda> migrate tools-elastic-03 to labvirt1009 [tools]
19:10 <yuvipanda> depooled tools nodes from labvirt1004 and 1007 [tools]
17:57 <yuvipanda> depool exec nodes on labvirt1002 [tools]
13:27 <chasemp> reboot tools-exec-1404 post depool for test [tools]
2016-10-31 §
21:50 <yuvipanda> deleted cyberbot queue with qconf -dq cyberbot [tools]
21:44 <yuvipanda> restarted cron on tools-cron-01 [tools]
2016-10-30 §
02:25 <yuvipanda> restarted maintain-kubeusers [tools]