1401-1450 of 10000 results (46ms)
2019-06-19 ยง
09:06 <akosiaris@puppetmaster1001> conftool action : set/pooled=yes; selector: name=kubernetes2002.* [production]
09:06 <akosiaris@puppetmaster1001> conftool action : set/pooled=yes; selector: name=kubernetes2003.* [production]
09:05 <ema@cumin1001> END (PASS) - Cookbook sre.hosts.upgrade-and-reboot (exit_code=0) [production]
09:03 <ema@cumin1001> START - Cookbook sre.hosts.upgrade-and-reboot [production]
08:57 <akosiaris> depool kubernetes200{2,3} for the same out discards investigation [production]
08:56 <ema@cumin1001> START - Cookbook sre.hosts.upgrade-and-reboot [production]
08:56 <akosiaris@puppetmaster1001> conftool action : set/pooled=no; selector: name=kubernetes2003.* [production]
08:56 <akosiaris@puppetmaster1001> conftool action : set/pooled=no; selector: name=kubernetes2002.* [production]
08:54 <akosiaris> uncordon kubernetes2001, reschedule some pods on it. Investigating out discards still [production]
08:51 <XioNoX> jnt push to codfw, remove old protect-old-lvs-servers term + update syslog target T224128 [production]
08:43 <ema@cumin1001> END (PASS) - Cookbook sre.hosts.upgrade-and-reboot (exit_code=0) [production]
08:43 <akosiaris> depool kubernetes2001 from all services to investigate some IP out discard statistics [production]
08:42 <akosiaris@puppetmaster1001> conftool action : set/pooled=no; selector: name=kubernetes2001.* [production]
08:36 <ema@cumin1001> END (PASS) - Cookbook sre.hosts.upgrade-and-reboot (exit_code=0) [production]
08:36 <akosiaris> cordon kubernetes2001 to investigate some IP out discard statistics [production]
08:34 <ema@cumin1001> START - Cookbook sre.hosts.upgrade-and-reboot [production]
08:28 <ema@cumin1001> START - Cookbook sre.hosts.upgrade-and-reboot [production]
08:24 <moritzm> installing new kernels with SACK fix on jessie servers [production]
08:21 <akosiaris> upgrade citoid, mathoid, termbox to latest chart releases to address the GC metric naming issue T220709 T222795 [production]
08:20 <akosiaris@deploy1001> scap-helm termbox finished [production]
08:20 <akosiaris@deploy1001> scap-helm termbox cluster staging completed [production]
08:20 <akosiaris@deploy1001> scap-helm termbox upgrade -f termbox-staging-values.yaml staging stable/termbox [namespace: termbox, clusters: staging] [production]
08:20 <akosiaris@deploy1001> scap-helm termbox finished [production]
08:20 <akosiaris@deploy1001> scap-helm termbox cluster codfw completed [production]
08:20 <akosiaris@deploy1001> scap-helm termbox cluster eqiad completed [production]
08:20 <akosiaris@deploy1001> scap-helm termbox upgrade -f termbox-values.yaml production stable/termbox [namespace: termbox, clusters: eqiad,codfw] [production]
08:19 <akosiaris@deploy1001> scap-helm mathoid finished [production]
08:18 <akosiaris@deploy1001> scap-helm mathoid cluster codfw completed [production]
08:18 <akosiaris@deploy1001> scap-helm mathoid cluster eqiad completed [production]
08:18 <akosiaris@deploy1001> scap-helm mathoid upgrade -f mathoid-values.yaml production stable/mathoid [namespace: mathoid, clusters: eqiad,codfw] [production]
08:14 <ema@cumin1001> END (PASS) - Cookbook sre.hosts.upgrade-and-reboot (exit_code=0) [production]
08:13 <akosiaris@deploy1001> scap-helm mathoid finished [production]
08:13 <akosiaris@deploy1001> scap-helm mathoid cluster staging completed [production]
08:13 <akosiaris@deploy1001> scap-helm mathoid upgrade -f mathoid-staging-values.yaml staging stable/mathoid [namespace: mathoid, clusters: staging] [production]
08:13 <akosiaris@deploy1001> scap-helm mathoid upgrade -f mathoid-staging-values.yaml staging stable/mathoid [namespace: mathoid, clusters: staging] [production]
08:13 <akosiaris@deploy1001> scap-helm citoid finished [production]
08:13 <akosiaris@deploy1001> scap-helm citoid cluster eqiad completed [production]
08:13 <akosiaris@deploy1001> scap-helm citoid upgrade -f citoid-eqiad-values.yaml production stable/citoid [namespace: citoid, clusters: eqiad] [production]
08:08 <ema@cumin1001> END (PASS) - Cookbook sre.hosts.upgrade-and-reboot (exit_code=0) [production]
08:07 <ema@cumin1001> START - Cookbook sre.hosts.upgrade-and-reboot [production]
08:02 <ema@cumin1001> START - Cookbook sre.hosts.upgrade-and-reboot [production]
08:01 <ema> cache nodes: resume rolling reboots for kernel and varnish upgrades T224694 [production]
08:00 <akosiaris@deploy1001> scap-helm citoid finished [production]
08:00 <akosiaris@deploy1001> scap-helm citoid cluster codfw completed [production]
08:00 <akosiaris@deploy1001> scap-helm citoid upgrade -f citoid-codfw-values.yaml production stable/citoid [namespace: citoid, clusters: codfw] [production]
08:00 <akosiaris@deploy1001> scap-helm citoid finished [production]
07:59 <akosiaris@deploy1001> scap-helm citoid cluster staging completed [production]
07:59 <akosiaris@deploy1001> scap-helm citoid upgrade -f citoid-staging-values.yaml staging stable/citoid [namespace: citoid, clusters: staging] [production]
07:56 <moritzm> rearmed keyholder on acmechief-test2001 [production]
07:51 <moritzm> installing vim security updates on stretch [production]