2701-2750 of 3029 results (17ms)
2016-05-28 §
21:21 <valhallasw`cloud> rebooting tools-exec-1204 (T136495) [tools]
2016-05-27 §
14:45 <YuviPanda> start moving tools-bastion-03 to use tools-puppetmaster-01 as puppetmaster [tools]
2016-05-25 §
20:15 <YuviPanda> deleted tools-bastion-mtemp per chasemp [tools]
19:43 <YuviPanda> delete devpi instance, not currently in use [tools]
19:39 <YuviPanda> run sudo dpkg --configure -a on tools-worker-1007 to get it unstuck [tools]
19:19 <YuviPanda> deleted tools-docker-builder-01 and -02, hosed hosts that are unused [tools]
17:18 <YuviPanda> fixed hhvm upgrade on tools-cron-01 [tools]
07:19 <YuviPanda> hard reboot tools-services-01, was completely stuck on /public/dumps [tools]
06:06 <bd808> Restarting all webservice jobs [tools]
05:33 <andrewbogott> rebooting tools-proxy-02 [tools]
2016-05-23 §
19:36 <YuviPanda> switched tools-checker to tools-checker-03 [tools]
13:28 <chasemp> 'apt-get install hhvm -y --force-yes' across trusty hosts to handle hhvm downgrade [tools]
2016-05-20 §
23:39 <bd808> Forced puppet run on bastion-02 & bastion-05 to apply fix for T135861 [tools]
19:44 <chasemp> tools-exec-1406 having issues rebooting [tools]
2016-05-19 §
21:07 <bd808> deployed jobutils 1.13 on bastions; now with '-l release=...' validation! [tools]
15:43 <YuviPanda> rebooting all tools worker instances [tools]
13:12 <chasemp> reboot tools-exec-1220 stuck in state of unresponsivenss [tools]
2016-05-13 §
00:40 <YuviPanda> cleared all queues that were in error state [tools]
2016-05-12 §
22:59 <YuviPanda> restart tools-worker-1004 to attempt bringing it back up [tools]
22:59 <YuviPanda> deploy k8s 1.2.4wmf1 on all proxy nodes [tools]
22:58 <YuviPanda> deploy k8s on all worker nodes [tools]
22:46 <YuviPanda> deploy k8s master for 1.2.4wmf1 [tools]
2016-05-10 §
04:25 <bd808> Added role::package::builder to tools-services-01 [tools]
2016-05-09 §
04:33 <YuviPanda> reboot tools-worker-1004, lots of ksoftirqd stuckness despite no actual containers running [tools]
2016-05-08 §
07:06 <YuviPanda> restarted admin tool [tools]
2016-05-05 §
13:11 <godog> cherry-pick https://gerrit.wikimedia.org/r/#/c/280652/ on puppetmaster [tools]
2016-04-28 §
04:15 <YuviPanda> delete half of the trusty webservice jobs [tools]
04:00 <YuviPanda> deleted all precise webservice jobs, waiting for webservicemonitor to bring them back up [tools]
2016-04-24 §
12:22 <YuviPanda> force deleted job 5435259 from pbbot per PeterBowman [tools]
2016-04-11 §
14:20 <andrewbogott> moving tools-bastion-mtemp to labvirt1009 [tools]
2016-04-06 §
15:20 <bd808> Removed local hack for T131906 from tools-puppetmaster-01 [tools]
2016-03-30 §
15:50 <andrewbogott> rebooting tools-proxy-01 in hopes of clearing some bad caches [tools]
2016-03-28 §
20:51 <yuvipanda> lifted RAM quota from 900Gigs to 1TB?! [tools]
20:30 <chasemp> change perm grant files from create-dbusers for chmod 400 chat chattr +i [tools]
2016-03-18 §
15:47 <chasemp> had to kill stalkboten as it was logging constant errors filling logs to the tune of hundreds of gigs [tools]
15:36 <chasemp> cleanup huge log collection for broken bot: /srv/project/tools/project/betacommand-dev/tspywiki/irc/logs# rm -fR SpamBotLog.log\.* [tools]
2016-03-11 §
20:57 <mutante> reverted font changes - puppet runs recovering [tools]
20:37 <mutante> more puppet issues due to font dependencies on trusty, on it [tools]
19:39 <mutante> should a tools-exec server be influenced by font packages on an mw appserver? [tools]
19:39 <mutante> fixed puppet runs on tools-exec (gerrit 276792) [tools]
2016-03-02 §
14:56 <chasemp> qdel 3956069 and 3758653 for abusing auth [tools]
2016-02-28 §
20:08 <bd808> Removed unwanted NFS mounts from tools-elastic-01.tools.eqiad.wmflabs [tools]
2016-02-26 §
19:08 <bd808> Upgraded Elasticsearch on tools-elastic-0[123] to 1.7.5 [tools]
2016-02-24 §
19:46 <chasemp> runonce deployed for https://gerrit.wikimedia.org/r/#/c/272891/ [tools]
2016-02-19 §
15:58 <chasemp> rerollout tools nfs shaping pilot for sanity in anticipation of formalization [tools]
09:21 <_joe_> killed cluebot3 instance on tools-exec-1207, writing 20 M/s to the error log [tools]
00:50 <yuvipanda> failover services to services-02 [tools]
2016-02-18 §
22:57 <valhallasw`cloud> restarted gridengine-master on tools-grid-master, otherwise all webservices will stay down [tools]
20:37 <yuvipanda> failover proxy back to tools-proxy-01 [tools]
19:46 <chasemp> repool labvirt1003 and depool labvirt1004 [tools]