201-250 of 1668 results (14ms)
2019-01-15 §
18:29 <bstorm_> T213711 installed python3-requests=2.11.1-1~bpo8+1 python3-urllib3=1.16-1~bpo8+1 on tools-proxy-03, which stopped the bleeding [tools]
14:55 <arturo> disable puppet in tools-docker-registry-01 and tools-docker-registry-02, trying with `role::wmcs::toolforge::docker::registry` in the puppetmaster for -03 and -04. The registry shouldn't be affected by this [tools]
14:21 <arturo> T213418 put a backup of the docker registry in NFS just in case: `aborrero@tools-docker-registry-02:$ sudo cp /srv/registry/registry.tar.gz /data/project/.system_sge/docker-registry-backup/` [tools]
2019-01-14 §
22:03 <bstorm_> T213711 Added UDP port needed for flannel packets to work to k8s worker sec groups in both eqiad and eqiad1-r [tools]
22:03 <bstorm_> T213711 Added ports needed for etcd-flannel to work on the etcd security group in eqiad [tools]
21:42 <zhuyifei1999_> also `write`-ed to them (as root). auth on my personal account would take a long time [tools]
21:37 <zhuyifei1999_> that command belonged to tools.scholia (with fnielsen as the ssh user) [tools]
21:36 <zhuyifei1999_> killed an egrep using too mush NFS bandwidth on tools-bastion-03 [tools]
21:33 <zhuyifei1999_> SIGTERM PID 12542 24780 875 14569 14722. `tail`s with parent as init, belonging to user maxlath. they should submit to grid. [tools]
16:44 <arturo> T213418 docker-registry.tools.wmflabs.org point floating IP to tools-docker-registry-02 [tools]
14:00 <arturo> T213421 disable updatetools in the new services nodes while building them [tools]
13:53 <arturo> T213421 delete tools-services-03/04 and create them with another prefix: tools-sge-services-03/04 to actually use the new role [tools]
13:47 <arturo> T213421 create tools-services-03 and tools-services-04 (stretch) they will use the new puppet role `role::wmcs::toolforge::services` [tools]
2019-01-11 §
11:55 <arturo> T213418 shutdown tools-docker-builder-05, will give a grace period before deleting the VM [tools]
10:51 <arturo> T213418 created tools-docker-builder-06 in eqiad1 [tools]
10:46 <arturo> T213418 migrating tools-docker-registry-02 from eqiad to eqiad1 [tools]
2019-01-10 §
22:45 <bstorm_> T213357 - Added 24 lighttpd nodes tot he new grid [tools]
18:54 <bstorm_> T213355 built and configured two more generic web nodes for the new grid [tools]
10:35 <gtirloni> deleted non-puppetized checks from tools-checker-{01,02} [tools]
00:12 <bstorm_> T213353 Added 36 exec nodes to the new grid [tools]
2019-01-09 §
20:16 <andrewbogott> moving tools-paws-worker-1013 and tools-paws-worker-1007 to eqiad1 [tools]
17:17 <andrewbogott> moving paws-worker-1017 and paws-worker-1016 to eqiad1 [tools]
14:42 <andrewbogott> experimentally moving tools-paws-worker-1019 to eqiad1 [tools]
09:59 <gtirloni> rebooted tools-checker-01 (T213252) [tools]
2019-01-07 §
17:21 <bstorm_> T67777 - set the max_u_jobs global grid config setting to 50 in the new grid [tools]
15:54 <bstorm_> T67777 Set stretch grid user job limit to 16 [tools]
05:45 <bd808> Manually installed python3-venv on tools-sgebastion-06. Gerrit patch submitted for proper automation. [tools]
2019-01-06 §
22:06 <bd808> Added floating ip to tools-sgebastion-06 (T212360) [tools]
2019-01-05 §
23:54 <bd808> Manually installed php-mbstring on tools-sgebastion-06. Gerrit patch submitted to install it on the rest of the Son of Grid Engine nodes. [tools]
2019-01-04 §
21:37 <bd808> Truncated /data/project/.system/accounting after archiving ~30 days of history [tools]
2019-01-03 §
21:03 <bd808> Enabled Puppet on tools-proxy-02 [tools]
20:53 <bd808> Disabled Puppet on tools-proxy-02 [tools]
20:51 <bd808> Enabled Puppet on tools-proxy-01 [tools]
20:49 <bd808> Disabled Puppet on tools-proxy-01 [tools]
2018-12-21 §
16:29 <andrewbogott> migrating tools-exec-1416 to labvirt1004 [tools]
16:01 <andrewbogott> moving tools-grid-master to labvirt1004 [tools]
00:35 <bd808> Installed tools-manifest 0.14 for T212390 [tools]
00:22 <bd808> Rebuiliding all docker containers with toollabs-webservice 0.43 for T212390 [tools]
00:19 <bd808> Installed toollabs-webservice 0.43 on all hosts for T212390 [tools]
00:01 <bd808> Installed toollabs-webservice 0.43 on tools-bastion-02 for T212390 [tools]
2018-12-20 §
20:43 <andrewbogott> moving moving tools-prometheus-02 to labvirt1004 [tools]
20:42 <andrewbogott> moving tools-k8s-etcd-02 to labvirt1003 [tools]
20:41 <andrewbogott> moving tools-package-builder-01 to labvirt1002 [tools]
2018-12-17 §
22:16 <bstorm_> Adding a bunch of hiera values and prefixes for the new grid - T212153 [tools]
19:18 <gtirloni> decreased nfs-mount-manager verbosity (T211817) [tools]
19:02 <arturo> T211977 add package tools-manifest 0.13 to stretch-tools & stretch-toolsbeta in aptly [tools]
13:46 <arturo> T211977 `aborrero@tools-services-01:~$ sudo aptly repo move trusty-tools stretch-toolsbeta 'tools-manifest (=0.12)'` [tools]
2018-12-11 §
13:18 <gtirloni> Removed BigBrother (T208357) [tools]
2018-12-05 §
12:17 <gtirloni> remoted node tools-worker-1029.tools.eqiad.wmflabs from cluster (T196973) [tools]
2018-12-04 §
22:47 <bstorm_> gtirloni added back main floating IP for tools-k8s-master-01 and removed unnecessary ones to stop k8s outage T164123 [tools]