2251-2300 of 10000 results (27ms)
2020-12-08 §
11:44 <akosiaris@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'apertium' for release 'production' . [production]
11:43 <akosiaris@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'apertium' for release 'staging' . [production]
11:43 <akosiaris@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'apertium' for release 'production' . [production]
11:42 <akosiaris@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'apertium' for release 'plain' . [production]
11:42 <akosiaris> add a TLS enabled release for apertium [production]
11:42 <akosiaris@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'apertium' for release 'production' . [production]
11:41 <akosiaris@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'apertium' for release 'staging' . [production]
11:41 <akosiaris@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'apertium' for release 'plain' . [production]
10:45 <ema> A:cp remove libvmod-tbf -- see https://gerrit.wikimedia.org/r/c/operations/puppet/+/623583/ [production]
10:33 <ema> cp4032: restart ats-{tls,be} [production]
10:29 <godog> reboot ms-be1021 - T268435 [production]
10:13 <ema> cp3054: upgrade varnish back to 6.0.7-1wm1 T264398 [production]
09:56 <ema> cp3054: downgrade varnish to 5.2.1-1wm1 T264398 [production]
09:06 <effie> upgrade scap to 3.16 everywhere - T268634 [production]
08:16 <godog> upgrade hw raid firmware on ms-be1032 and reboot - T269624 [production]
08:09 <godog> power reset ms-be1032 [production]
03:25 <ryankemper@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
03:16 <ryankemper@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
03:05 <ryankemper@cumin2001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
03:03 <ryankemper@cumin2001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
02:30 <ejegg> deployed queue consumers for new Adyen capture jobs [production]
01:57 <ryankemper@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
01:56 <ryankemper@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
01:55 <ryankemper@cumin2001> START - Cookbook sre.wdqs.data-transfer [production]
01:54 <ryankemper@cumin2001> START - Cookbook sre.wdqs.data-transfer [production]
01:53 <ryankemper@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
01:52 <ryankemper@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
01:51 <ryankemper> T246345 Brought two new public wdqs nodes `wdqs101[2,3]` into service: `sudo confctl select 'name=wdqs1012.eqiad.wmnet|wdqs1013.eqiad.wmnet' set/pooled=yes:weight=10` [production]
01:50 <ryankemper@mwmaint1002> conftool action : set/pooled=yes:weight=10; selector: name=wdqs1012.eqiad.wmnet|wdqs1013.eqiad.wmnet [production]
01:49 <ryankemper@cumin2001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
01:49 <ryankemper@cumin2001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
01:49 <ryankemper@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
01:44 <ryankemper@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
01:43 <ryankemper@cumin2001> START - Cookbook sre.wdqs.data-transfer [production]
01:43 <ryankemper@cumin2001> START - Cookbook sre.wdqs.data-transfer [production]
01:43 <ryankemper@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
01:43 <ryankemper@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
01:23 <ryankemper@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
01:21 <ryankemper@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
01:18 <ryankemper@cumin2001> START - Cookbook sre.hosts.downtime [production]
01:18 <ryankemper@cumin2001> START - Cookbook sre.hosts.downtime [production]
01:15 <ryankemper@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
01:13 <ryankemper@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
01:12 <ryankemper@cumin1001> START - Cookbook sre.hosts.downtime [production]
01:11 <ryankemper@cumin1001> START - Cookbook sre.hosts.downtime [production]
00:56 <ryankemper> T269204 reimaging the following instances to debian buster: `eqiad public`->`wdqs1007`, `codfw public`->`wdqs2007`, `codfw internal`->`wdqs2008`, `test`->`wdqs1010` [production]
2020-12-07 §
23:44 <eileen> process-control config revision is b3bc1959cd [production]
23:13 <eileen> process-control config revision is f48cdb9184 [production]
23:13 <ryankemper@cumin2001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
23:12 <ryankemper@cumin2001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]