451-500 of 943 results (13ms)
2019-03-11 §
00:53 <bd808> Re-enabled 13 queue instances that had been disabled by LDAP failures during job initialization (T217280) [tools]
2019-03-08 §
19:48 <andrewbogott> repooling tools-exec-1430 and tools-sgeexec-0905 to compare ldap usage [tools]
19:21 <andrewbogott> depooling tools-exec-1430 and tools-sgeexec-0905 to compare ldap usage [tools]
17:49 <bd808> Re-enabled 4 queue instances that had been disabled by LDAP failures during job initialization (T217280) [tools]
00:30 <bd808> DNS record created for trusty-dev.tools.wmflabs.org (Trusty secondary bastion) [tools]
2019-03-07 §
23:31 <bd808> Updated DNS to point login.tools.wmflabs.org at 185.15.56.48 (Stretch bastion) [tools]
04:15 <bd808> Killed 3 orphan processes on Trusty grid [tools]
04:01 <bd808> Cleared error state on a large number of Stretch grid queues which had been disabled by LDAP and/or NFS hiccups (T217280) [tools]
2019-03-04 §
14:05 <gtirloni> rebooted tools-docker-registry-{03,04}, tools-puppetmaster-02 and tools-puppetdb-01 (load avg >45, not accessible) [tools]
2019-02-27 §
19:34 <andrewbogott> uncordoning tools-worker-1028, 1002 and 1005, now in eqiad1-r [tools]
01:35 <bd808> Shutdown tools-webgrid-lighttpd-1419.tools.eqiad.wmflabs via horizon (T217152) [tools]
01:29 <bd808> Depooled tools-webgrid-lighttpd-1419.tools.eqiad.wmflabs (T217152) [tools]
01:26 <bd808> Disabled job queues and rescheduled continuous jobs away from tools-exec-14{33,34,35,36,37,38,39,40,41,42}.tools.eqiad.wmflabs (T217152) [tools]
2019-02-26 §
19:01 <gtirloni> pushed updated docker images [tools]
17:30 <andrewbogott> draining and cordoning tools-worker-1027 for a region migration test [tools]
2019-02-25 §
23:20 <bstorm_> Depooled tools-sgeexec-0914 and tools-sgeexec-0915 for T217066 [tools]
12:21 <chicocvenancio> PAWS: killed proxy and hub pods to attempt to get it to see routes to open notebooks servers to no avail. Restarted BernhardHumm's notebook pod T217010 [tools]
07:44 <zhuyifei1999_> I saved dmesg and process list to a few files in /root if that helps debugging [tools]
2019-02-22 §
16:29 <gtirloni> upgraded and rebooted tools-puppetmaster-01 (new kernel) [tools]
2019-02-20 §
23:30 <zhuyifei1999_> begin rebuilding all docker images T178601 T193646 T215683 [tools]
15:03 <andrewbogott> moving tools-exec-1413 and tools-exec-1442 [tools]
2019-02-19 §
01:49 <bd808> Revoked Toolforge project membership for user DannyS712 (T215092) [tools]
2019-02-18 §
20:45 <gtirloni> upgraded and rebooted tools-sgebastion-07 (login-stretch) [tools]
2019-02-16 §
04:52 <zhuyifei1999_> copied the resolv.conf from tools-k8s-master-01, removing secondary DNS to make sure puppet fixes that, and starting puppet [tools]
04:31 <zhuyifei1999_> then started nslcd vis systemctl and `id zhuyifei1999` returns correct stuffs [tools]
04:30 <zhuyifei1999_> `nslcd -nd` complains about 'nslcd: bind() to /var/run/nslcd/socket failed: Address already in use'. SIGTERMed a background nslcd, `rmdir /var/run/nslcd/socket`, and `nslcd -nd` seemingly starts to work [tools]
2019-02-14 §
21:57 <bd808> Deleted old tools-proxy-02 instance [tools]
21:57 <bd808> Deleted old tools-proxy-01 instance [tools]
21:56 <bd808> Deleted old tools-package-builder-01 instance [tools]
19:55 <andrewbogott> moving tools-webgrid-generic-1401 and tools-webgrid-lighttpd-1419 [tools]
17:35 <arturo> T215154 tools-sgebastion-07 now running systemd 239 and starts enforcing user limits [tools]
2019-02-12 §
01:24 <bd808> Stopped maintain-kubeusers, edited /etc/kubernetes/tokenauth, restarted maintain-kubeusers (T215704) [tools]
2019-02-11 §
22:57 <bd808> Shutoff tools-webgrid-lighttpd-14{01,13,24,26,27,28} via Horizon UI [tools]
22:34 <bd808> Decommissioned tools-webgrid-lighttpd-14{01,13,24,26,27,28} [tools]
22:23 <bd808> sudo exec-manage depool tools-webgrid-lighttpd-1401.tools.eqiad.wmflabs [tools]
22:21 <bd808> sudo exec-manage depool tools-webgrid-lighttpd-1413.tools.eqiad.wmflabs [tools]
22:18 <bd808> sudo exec-manage depool tools-webgrid-lighttpd-1428.tools.eqiad.wmflabs [tools]
22:07 <bd808> sudo exec-manage depool tools-webgrid-lighttpd-1427.tools.eqiad.wmflabs [tools]
22:06 <bd808> sudo exec-manage depool tools-webgrid-lighttpd-1424.tools.eqiad.wmflabs [tools]
22:05 <bd808> sudo exec-manage depool tools-webgrid-lighttpd-1426.tools.eqiad.wmflabs [tools]
20:06 <bstorm_> Ran apt-get clean on tools-sgebastion-07 since it was running out of disk (and lots of it was the apt cache) [tools]
19:08 <bd808> Upgraded tools-manifest on tools-cron-01 to v0.19 (T107878) [tools]
18:57 <bd808> Upgraded tools-manifest on tools-sgecron-01 to v0.19 (T107878) [tools]
18:57 <bd808> Built tools-manifest_0.19_all.deb and published to aptly repos (T107878) [tools]
18:26 <bd808> Upgraded tools-manifest on tools-sgecron-01 to v0.18 (T107878) [tools]
18:25 <bd808> Built tools-manifest_0.18_all.deb and published to aptly repos (T107878) [tools]
18:12 <bd808> Upgraded tools-manifest on tools-sgecron-01 to v0.17 (T107878) [tools]
18:08 <bd808> Built tools-manifest_0.17_all.deb and published to aptly repos (T107878) [tools]
10:41 <godog> flip tools-prometheus proxy back to tools-prometheus-01 and upgrade to prometheus 2.7.1 [tools]
2019-02-08 §
01:07 <bd808> Creating tools-sgebastion-07 [tools]