|
2026-10-28
ยง
|
| 20:50 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1343-1356].eqiad.wmnet |
[production] |
| 20:50 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1329-1342].eqiad.wmnet |
[production] |
| 20:50 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1329-1342].eqiad.wmnet |
[production] |
| 20:42 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1329-1342].eqiad.wmnet |
[production] |
| 20:40 |
<sbassett> |
Deployed security mitigations for T435455, T439844, T442538 |
[production] |
| 20:29 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1329-1342].eqiad.wmnet |
[production] |
| 20:29 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1315-1328].eqiad.wmnet |
[production] |
| 20:29 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1315-1328].eqiad.wmnet |
[production] |
| 20:20 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1315-1328].eqiad.wmnet |
[production] |
| 20:19 |
<mutante> |
etherpad1004 - apache httpd disabled by puppet - etherpad running normally |
[production] |
| 20:17 |
<arlolra@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1358832|Turn off "report a visual" bug on wikipedias (T430970)]] (duration: 11m 26s) |
[production] |
| 20:14 |
<cmooney@cumin1004> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 20:14 |
<cmooney@cumin1004> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update last few IPs that got awawy - cmooney@cumin1004" |
[production] |
| 20:14 |
<cmooney@cumin1004> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update last few IPs that got awawy - cmooney@cumin1004" |
[production] |
| 20:12 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1315-1328].eqiad.wmnet |
[production] |
| 20:12 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1301-1314].eqiad.wmnet |
[production] |
| 20:12 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1301-1314].eqiad.wmnet |
[production] |
| 20:11 |
<arlolra@deploy1003> |
arlolra: Continuing with deployment |
[production] |
| 20:10 |
<arlolra@deploy1003> |
arlolra: Backport for [[gerrit:1358832|Turn off "report a visual" bug on wikipedias (T430970)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 20:10 |
<cmooney@cumin1004> |
START - Cookbook sre.dns.netbox |
[production] |
| 20:06 |
<arlolra@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1358832|Turn off "report a visual" bug on wikipedias (T430970)]] |
[production] |
| 20:03 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1301-1314].eqiad.wmnet |
[production] |
| 20:03 |
<cmooney@cumin1004> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 20:03 |
<cmooney@cumin1004> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update subint entries for row B interfaces on eqiad CRs - cmooney@cumin1004" |
[production] |
| 20:03 |
<cmooney@cumin1004> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update subint entries for row B interfaces on eqiad CRs - cmooney@cumin1004" |
[production] |
| 19:59 |
<cmooney@cumin1004> |
START - Cookbook sre.dns.netbox |
[production] |
| 19:55 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1301-1314].eqiad.wmnet |
[production] |
| 19:54 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1284-1289,1291-1297,1299].eqiad.wmnet |
[production] |
| 19:54 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1284-1289,1291-1297,1299].eqiad.wmnet |
[production] |
| 19:54 |
<cmooney@cumin1004> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 19:54 |
<cmooney@cumin1004> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update subint entries for row B interfaces on eqiad CRs - cmooney@cumin1004" |
[production] |
| 19:54 |
<cmooney@cumin1004> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update subint entries for row B interfaces on eqiad CRs - cmooney@cumin1004" |
[production] |
| 19:52 |
<akhatun@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1359949|Prefetch the LAC revision_feature_counts artifact on group1 wikipedias ]] (duration: 11m 48s) |
[production] |
| 19:52 |
<pt1979@cumin1004> |
START - Cookbook sre.hosts.dhcp for host ml-serve1016.eqiad.wmnet (live debug) |
[production] |
| 19:49 |
<cmooney@cumin1004> |
START - Cookbook sre.dns.netbox |
[production] |
| 19:46 |
<akhatun@deploy1003> |
akhatun, otto: Continuing with deployment |
[production] |
| 19:45 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1284-1289,1291-1297,1299].eqiad.wmnet |
[production] |
| 19:45 |
<akhatun@deploy1003> |
akhatun, otto: Backport for [[gerrit:1359949|Prefetch the LAC revision_feature_counts artifact on group1 wikipedias ]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 19:40 |
<akhatun@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1359949|Prefetch the LAC revision_feature_counts artifact on group1 wikipedias ]] |
[production] |
| 19:37 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1284-1289,1291-1297,1299].eqiad.wmnet |
[production] |
| 19:37 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1269-1283].eqiad.wmnet |
[production] |
| 19:37 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1269-1283].eqiad.wmnet |
[production] |
| 19:30 |
<akhatun@deploy1003> |
helmfile [codfw] DONE helmfile.d/services/lac-lambdas: apply |
[production] |
| 19:29 |
<akhatun@deploy1003> |
helmfile [codfw] START helmfile.d/services/lac-lambdas: apply |
[production] |
| 19:28 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1269-1283].eqiad.wmnet |
[production] |
| 19:26 |
<akhatun@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/lac-lambdas: apply |
[production] |
| 19:26 |
<akhatun@deploy1003> |
helmfile [eqiad] START helmfile.d/services/lac-lambdas: apply |
[production] |
| 19:23 |
<akhatun@deploy1003> |
helmfile [staging] DONE helmfile.d/services/lac-lambdas: apply |
[production] |
| 19:22 |
<akhatun@deploy1003> |
helmfile [staging] START helmfile.d/services/lac-lambdas: apply |
[production] |
| 19:21 |
<inflatador> |
[bking@deploy1003] `curl -XPOST -u $U -s https://opensearch-ttmserver.svc.eqiad.wmnet:30443/_cluster/reroute` Fix yellow status |
[production] |