351-400 of 614 results (23ms)
2020-06-11 §
11:32 <arturo> apparently every python script segfaults in toolsbeta-puppetmaster-03 [toolsbeta]
11:27 <arturo> puppetdb wasn't the problem. The problem is puppet-enc segfaulting in toolsbeta-puppetmaster-03 [toolsbeta]
11:21 <arturo> puppet not working bc puppetdb, run `aborrero@toolsbeta-puppetdb-02:~ $ sudo systemctl restart puppetdb` [toolsbeta]
2020-06-04 §
21:06 <andrewbogott> added krenair to toolsbeta.admin group in ldap [toolsbeta]
2020-05-28 §
11:27 <arturo> cleanup livehackings [toolsbeta]
10:31 <arturo> livehacking puppetmaster and toolsbeta-proxy-1 to test https://gerrit.wikimedia.org/r/c/operations/puppet/+/599139 (T253816) [toolsbeta]
10:30 <arturo> livehacking puppetmaster to test https://gerrit.wikimedia.org/r/c/operations/puppet/+/599139 [toolsbeta]
2020-05-27 §
12:02 <arturo> the k8s cluster is now running v1.16.10 (T246122) [toolsbeta]
11:05 <arturo> trying `modules/kubeadm/files/wmcs-k8s-node-upgrade.py --control toolsbeta-test-k8s-control-1 --project toolsbeta --domain eqiad.wmflabs --src-version 1.15 --dst-version 1.16.10 -n toolsbeta-test-k8s-worker-1 -n toolsbeta-test-k8s-worker-2 -n toolsbeta-test-k8s-worker-3` (T246122) [toolsbeta]
11:02 <arturo> upgraded the rest of the k8s control plane nodes to 1.16.10 (T246122) [toolsbeta]
10:58 <arturo> running `aborrero@toolsbeta-test-k8s-control-1:~ $ sudo apt-get install kubelet -y` in the 1.16 version from the component repo (T246122) [toolsbeta]
10:58 <arturo> running `aborrero@toolsbeta-test-k8s-control-1:~ $ sudo -i kubeadm upgrade apply v1.16.10` and this time it works! (T246122) [toolsbeta]
2020-05-26 §
16:17 <bstorm_> fix incorrect volume name in kubeadm-config T246122 [toolsbeta]
15:02 <arturo> first k8s upgrade failed for yet-to-be-known reasons (T246122) [toolsbeta]
14:54 <arturo> `aborrero@toolsbeta-test-k8s-control-1:~ $ sudo -i kubeadm upgrade apply v1.16.10` (T246122) [toolsbeta]
14:54 <arturo> bump installed version of kubeadm and kubectl to 1.16.10 (T246122) [toolsbeta]
09:57 <arturo> installing kubectl/kubeadm 1.16.9 on k8s worker nodes (T246122) [toolsbeta]
09:56 <arturo> installing kubectl/kubeadm 1.16.9 on k8s control nodes (T246122) [toolsbeta]
09:30 <arturo> set `profile::wmcs::kubeadm::component: 'thirdparty/kubeadm-k8s-1-16'` at project level for trying T246122 [toolsbeta]
09:25 <arturo> `aborrero@toolsbeta-puppetdb-02:~ $ sudo systemctl restart puppetdb` broken puppet in this project because puppetdb is down again [toolsbeta]
2020-05-21 §
22:14 <bd808> Building tools-webservice 0.70 via wmcs-package-build.py [toolsbeta]
2020-05-19 §
12:20 <arturo> trying to install tesseract 4.1.0 in toolsbeta-sgebastion-04 (T247422) [toolsbeta]
10:18 <arturo> `aborrero@toolsbeta-puppetdb-02:~$ sudo systemctl restart puppetdb` [toolsbeta]
2020-05-15 §
20:48 <bstorm_> found an error in the new version of maintain-kubeusers, removing the deployment for now T246059 [toolsbeta]
20:35 <bstorm_> updating the maintain-kubeusers image to be able to control admin accounts [toolsbeta]
2020-05-14 §
12:09 <arturo> created puppet prefix toolsbeta-acme-chief in horizon (T252762) [toolsbeta]
12:08 <arturo> created toolsbeta-acme-chief-01 VM (T252762) [toolsbeta]
2020-05-12 §
18:35 <bstorm_> upgraded to using typha and rolled back to not doing so -- no affect on existing network T250863 [toolsbeta]
17:44 <bstorm_> set the calico version to v3.14.0 because the new liveness probe isn't compatible with the old version. T250863 [toolsbeta]
17:35 <bstorm_> deployed an updated bit of yaml for calico without upgrading the version first T250863 [toolsbeta]
2020-05-08 §
12:48 <arturo> allocated floating IP `185.15.56.12` for the VM `toolsbeta-email-01` and FQDN `mail.toolsbeta.wmflabs.org` (T120225) [toolsbeta]
12:24 <arturo> added puppet prefix `toolsbeta-email` (T120225) [toolsbeta]
2020-05-07 §
16:33 <arturo> livehack toolsbeta-puppetmaster-03 with https://gerrit.wikimedia.org/r/c/operations/puppet/+/594945 (T251297 and T250866) [toolsbeta]
12:36 <arturo> cleanup livehacks in toolsbeta-puppetmaster-03 [toolsbeta]
11:12 <arturo> livehack toolsbeta-puppetmaster-03 with https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/594925 and https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/594926 (T251297 and T250866) [toolsbeta]
2020-05-06 §
19:11 <bstorm_> updated toollabs-webservice to 0.69 for toolsbeta [toolsbeta]
09:58 <arturo> livehacking toolsbeta-puppetmaster-03 with https://gerrit.wikimedia.org/r/c/operations/puppet/+/594471 (T251297) [toolsbeta]
2020-05-05 §
10:04 <arturo> add herron as user and projectadmin, we will work on the email setup (T120225) [toolsbeta]
09:59 <arturo> created VM toolsbeta-mail-01 (T120225) [toolsbeta]
2020-05-04 §
13:02 <arturo> `aborrero@toolsbeta-puppetdb-02:~ $ sudo systemctl restart puppetdb.service` trying to bring back puppetdb, which is preventing puppet agent runs in the whole project [toolsbeta]
2020-04-29 §
19:48 <bstorm_> ran the scary rewrite-psp-preset.sh script across toolsbeta T247455 [toolsbeta]
2020-04-20 §
14:47 <arturo> added joakino to toolsbeta.admin LDAP group [toolsbeta]
12:06 <arturo> installing tools-webservice v0.68 for testing [toolsbeta]
11:05 <arturo> poweroff `toolsbeta-services-01`. I suspect this VM is not in use because no puppet role is in used there [toolsbeta]
10:58 <arturo> run `aborrero@toolsbeta-puppetdb-02:~ $ sudo systemctl restart puppetdb` the service was in failed state, causing puppet failures across the whole project [toolsbeta]
2020-04-10 §
19:32 <bstorm_> deployed webservice 0.67 T249843 [toolsbeta]
18:59 <bstorm_> delete toolsbeta-gitlab-01 and build toolsbeta-workflow-test T249946 [toolsbeta]
00:40 <bd808> REbooting toolsbeta-sgebastion-04. NFS seemed messed up [toolsbeta]
2020-04-08 §
01:10 <bstorm_> upgrade toollabs-webservice to 0.66 for qa T249390 [toolsbeta]
2020-03-31 §
23:39 <bstorm_> deployed toollabs-webservice-0.65 to toolsbeta [toolsbeta]