2101-2150 of 3208 results (24ms)
2018-01-27 §
00:55 <arturo> at tools-static-11 the kernel OOM killer stopped git gc at about 20% :-( [tools]
00:25 <arturo> (/srv is almost full) aborrero@tools-static-11:/srv/cdnjs$ sudo git gc --aggressive [tools]
2018-01-25 §
23:47 <arturo> fix last deprecation warnings in tools-elastic-03, tools-elastic-02, tools-proxy-01 and tools-proxy-02 by replacing by hand configtimeout with http_configtimeout in /etc/puppet/puppet.conf [tools]
23:20 <arturo> T179386 aborrero@tools-clushmaster-01:~$ clush -w @all 'sudo puppet agent -t -v' [tools]
05:25 <arturo> deploying misctools and jobutils 1.29 for T179386 [tools]
2018-01-23 §
19:41 <madhuvishy> Add bstorm to project admins [tools]
15:48 <bd808> Admin clean up; removed Coren, Ryan Lane, and Springle. [tools]
14:17 <chasemp> add me, arturo, chico to sudoers and removed marc [tools]
2018-01-22 §
18:32 <arturo> T181948 T185314 deploying jobutils and misctools v1.28 in the cluster [tools]
11:21 <arturo> puppet in the cluster is mostly fine, except for a couple of deprecation warnings, a conn timeout to services-01 and https://phabricator.wikimedia.org/T181948#3916790 [tools]
10:31 <arturo> aborrero@tools-clushmaster-01:~$ clush -w @all 'sudo puppet agent -t -v' <--- check again how is the cluster with puppet [tools]
10:18 <arturo> T181948 deploy misctools 1.27 in the cluster [tools]
2018-01-19 §
17:32 <arturo> T185314 deploying new version of jobutils 1.27 [tools]
12:56 <arturo> the puppet status across the fleet seems good, only minor things like T185314 , T179388 and T179386 [tools]
12:39 <arturo> aborrero@tools-clushmaster-01:~$ clush -w @all 'sudo puppet agent -t -v' [tools]
2018-01-18 §
16:11 <arturo> aborrero@tools-clushmaster-01:~$ sudo aptitude purge vblade vblade-persist runit (for something similar to T182781) [tools]
15:42 <arturo> T178717 aborrero@tools-clushmaster-01:~$ clush -w @all 'sudo puppet agent -t -v' [tools]
13:52 <arturo> T178717 aborrero@tools-clushmaster-01:~$ clush -f 1 -w @all 'sudo facter | grep lsbdistcodename | grep trusty && sudo apt-upgrade trusty-wikimedia -v' [tools]
13:44 <chasemp> upgrade wikimedia packages on tools-bastion-05 [tools]
12:24 <arturo> T178717 aborrero@tools-exec-1401:~$ sudo apt-upgrade trusty-wikimedia -v [tools]
12:11 <arturo> T178717 aborrero@tools-webgrid-generic-1402:~$ sudo apt-upgrade trusty-wikimedia -v [tools]
11:42 <arturo> T178717 aborrero@tools-clushmaster-01:~$ clush -w @all 'sudo puppet agent --test' [tools]
2018-01-17 §
18:47 <arturo> aborrero@tools-clushmaster-01:~$ clush -w @all 'apt-show-versions | grep upgradeable | grep trusty-wikimedia' | tee pending-upgrades-report-trusty-wikimedia.txt [tools]
17:55 <arturo> aborrero@tools-clushmaster-01:~$ clush -w @all 'sudo report-pending-upgrades -v' | tee pending-upgrades-report.txt [tools]
15:15 <andrewbogott> running purge-old-kernels on all Trusty exec nodes [tools]
15:15 <andrewbogott> repooling exec-manage tools-exec-1430. [tools]
15:04 <andrewbogott> depooling exec-manage tools-exec-1430. Experimenting with purge-old-kernels [tools]
14:09 <arturo> T181647 aborrero@tools-clushmaster-01:~$ clush -w @all 'sudo puppet agent --test' [tools]
2018-01-16 §
22:01 <chasemp> qstat -explain E -xml | grep 'name' | sed 's/<name>//' | sed 's/<\/name>//' | xargs qmod -cq [tools]
21:54 <chasemp> tools-exec-1436:~$ /sbin/reboot [tools]
21:24 <andrewbogott> repooled tools-exec-1420 and tools-webgrid-lighttpd-1417 [tools]
21:14 <andrewbogott> depooling tools-exec-1420 and tools-webgrid-lighttpd-1417 [tools]
20:58 <andrewbogott> depooling tools-exec-1412, 1415, 1417, tools-webgrid-lighttpd-1415, 1416, 1422, 1426 [tools]
20:56 <andrewbogott> repooling tools-exec-1416, 1418, 1424, tools-webgrid-lighttpd-1404, tools-webgrid-lighttpd-1410 [tools]
20:46 <andrewbogott> depooling tools-exec-1416, 1418, 1424, tools-webgrid-lighttpd-1404, tools-webgrid-lighttpd-1410 [tools]
20:46 <andrewbogott> repooled tools-exec-1406, 1421, 1436, 1437, tools-webgrid-generic-1404, tools-webgrid-lighttpd-1409, tools-webgrid-lighttpd-1411, tools-webgrid-lighttpd-1418, tools-webgrid-lighttpd-1425 [tools]
20:33 <andrewbogott> depooling tools-exec-1406, 1421, 1436, 1437, tools-webgrid-generic-1404, tools-webgrid-lighttpd-1409, tools-webgrid-lighttpd-1411, tools-webgrid-lighttpd-1418, tools-webgrid-lighttpd-1425 [tools]
20:20 <andrewbogott> depooling tools-webgrid-lighttpd-1412 and tools-exec-1423 for host reboot [tools]
20:19 <andrewbogott> repooled tools-exec-1409, 1410, 1414, 1419, 1427, 1428 tools-webgrid-generic-1401, tools-webgrid-lighttpd-1406 [tools]
20:02 <andrewbogott> depooling tools-exec-1409, 1410, 1414, 1419, 1427, 1428 tools-webgrid-generic-1401, tools-webgrid-lighttpd-1406 [tools]
20:00 <andrewbogott> depooled and repooled tools-webgrid-lighttpd-1427 tools-webgrid-lighttpd-1428 tools-exec-1413 tools-exec-1442 for host reboot [tools]
18:50 <andrewbogott> switched active proxy back to tools-proxy-02 [tools]
18:50 <andrewbogott> repooling tools-exec-1422 and tools-webgrid-lighttpd-1413 [tools]
18:34 <andrewbogott> moving proxy from tools-proxy-02 to tools-proxy-01 [tools]
18:31 <andrewbogott> depooling tools-exec-1422 and tools-webgrid-lighttpd-1413 for host reboot [tools]
18:26 <andrewbogott> repooling tools-exec-1404 and 1434 for host reboot [tools]
18:06 <andrewbogott> depooling tools-exec-1404 and 1434 for host reboot [tools]
18:04 <andrewbogott> repooling tools-exec-1402, 1426, 1429, 1433, tools-webgrid-lighttpd-1408, 1414, 1424 [tools]
17:48 <andrewbogott> depooling tools-exec-1402, 1426, 1429, 1433, tools-webgrid-lighttpd-1408, 1414, 1424 [tools]
17:28 <andrewbogott> disabling tools-webgrid-generic-1402, tools-webgrid-lighttpd-1403, tools-exec-1403 for host reboot [tools]