2020-12-17
§
|
21:42 |
<bstorm> |
doing the same procedure to increase the timeouts more T267966 |
[tools] |
19:56 |
<bstorm> |
puppet enabled one at a time, letting things catch up. Timeouts are now adjusted to something closer to fsync values T267966 |
[tools] |
19:44 |
<bstorm> |
set etcd timeouts seed value to 20 instead of the default 10 (profile::wmcs::kubeadm::etcd_latency_ms) T267966 |
[tools] |
18:58 |
<bstorm> |
disabling puppet on k8s-etcd servers to alter the timeouts T267966 |
[tools] |
14:23 |
<arturo> |
regenerating puppet cert with proper alt names in tools-k8s-etcd-4 (T267966) |
[tools] |
14:21 |
<arturo> |
regenerating puppet cert with proper alt names in tools-k8s-etcd-5 (T267966) |
[tools] |
14:18 |
<arturo> |
regenerating puppet cert with proper alt names in tools-k8s-etcd-6 (T267966) |
[tools] |
14:17 |
<arturo> |
regenerating puppet cert with proper alt names in tools-k8s-etcd-7 (T267966) |
[tools] |
14:15 |
<arturo> |
regenerating puppet cert with proper alt names in tools-k8s-etcd-8 (T267966) |
[tools] |
14:12 |
<arturo> |
updated kube-apiserver manifest with new etcd nodes (T267966) |
[tools] |
13:56 |
<arturo> |
adding etcd dns_alt_names hiera keys to the puppet prefix https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-puppet/+/beb27b45a74765a64552f2d4f70a40b217b4f4e9%5E%21/ |
[tools] |
13:12 |
<arturo> |
making k8s api server aware of the new etcd nodes via hiera update https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-puppet/+/3761c4c4dab1c3ed0ab0a1133d2ccf3df6c28baf%5E%21/ (T267966) |
[tools] |
12:54 |
<arturo> |
joining new etcd nodes in the k8s etcd cluster (T267966) |
[tools] |
12:52 |
<arturo> |
adding more etcd nodes in the hiera key in tools-k8s-etcd puppet prefix https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-puppet/+/b4f60768078eccdabdfab4cd99c7c57076de51b2 |
[tools] |
12:50 |
<arturo> |
dropping more unused hiera keys in the tools-k8s-etcd puppet prefix https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-puppet/+/e9e66a6787d9b91c08cf4742a27b90b3e6d05aac |
[tools] |
12:49 |
<arturo> |
dropping unused hiera keys in the tools-k8s-etcd puppet prefix https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-puppet/+/2b4cb4a41756e602fb0996e7d0210e9102172424 |
[tools] |
12:16 |
<arturo> |
created VM `tools-k8s-etcd-8` (T267966) |
[tools] |
12:15 |
<arturo> |
created VM `tools-k8s-etcd-7` (T267966) |
[tools] |
12:13 |
<arturo> |
created `tools-k8s-etcd` anti-affinity server group |
[tools] |
2020-12-11
§
|
18:29 |
<bstorm> |
certificatesigningrequest.certificates.k8s.io "tool-production-error-tasks-metrics" deleted to stop maintain-kubeusers issues |
[tools] |
12:14 |
<dcaro> |
upgrading stable/main (clinic duty) |
[tools] |
12:12 |
<dcaro> |
upgrading buster-wikimedia/main (clinic duty) |
[tools] |
12:03 |
<dcaro> |
upgrading stable-updates/main, mainly cacertificates (clinic duty) |
[tools] |
12:01 |
<dcaro> |
upgrading stretch-backports/main, mainly libuv (clinic duty) |
[tools] |
11:58 |
<dcaro> |
disabled all the repos blocking upgrades on tools-package-builder-02 (duplicated, other releases...) |
[tools] |
11:35 |
<arturo> |
uncordon tools-k8s-worker-71 and tools-k8s-worker-55, they weren't uncordoned yesterday for whatever reasons (T263284) |
[tools] |
11:27 |
<dcaro> |
upgrading stretch-wikimedia/main (clinic duty) |
[tools] |
11:20 |
<dcaro> |
upgrading stretch-wikimedia/thirdparty/mono-project-stretch (clinic duty) |
[tools] |
11:08 |
<dcaro> |
upgrade stretch-wikimedia/component/php72 (minor upgrades) (clinic duty) |
[tools] |
11:04 |
<dcaro> |
upgrade oldstable/main packages (clinic duty) |
[tools] |
10:58 |
<dcaro> |
upgrade kubectl done (clinic duty) |
[tools] |
10:53 |
<dcaro> |
upgrade kubectl (clinic duty) |
[tools] |
10:16 |
<dcaro> |
upgrading oldstable/main packages (clinic duty) |
[tools] |