|
2026-10-15
§
|
| 05:02 |
<oblivian@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2291.codfw.wmnet |
[production] |
| 05:02 |
<oblivian@cumin1004> |
START - Cookbook sre.k8s.enable_virtualization rolling provision on D{wikikube-worker[2291-2320].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) |
[production] |
| 04:55 |
<marostegui@cumin1004> |
START - Cookbook sre.mysql.sanitize-wiki Managing sanitization for wikis mlwikivoyage in section s5 |
[production] |
| 04:54 |
<oblivian@cumin1004> |
START - Cookbook sre.hosts.provision for host wikikube-worker1314.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL |
[production] |
| 04:53 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1314.eqiad.wmnet |
[production] |
| 04:52 |
<oblivian@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1314.eqiad.wmnet |
[production] |
| 04:52 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1313.eqiad.wmnet |
[production] |
| 04:52 |
<oblivian@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1313.eqiad.wmnet |
[production] |
| 04:52 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1313.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL |
[production] |
| 04:43 |
<oblivian@cumin1004> |
START - Cookbook sre.hosts.provision for host wikikube-worker1313.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL |
[production] |
| 04:43 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1313.eqiad.wmnet |
[production] |
| 04:42 |
<oblivian@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1313.eqiad.wmnet |
[production] |
| 04:42 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1312.eqiad.wmnet |
[production] |
| 04:40 |
<oblivian@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1312.eqiad.wmnet |
[production] |
| 04:39 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1312.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL |
[production] |
| 04:30 |
<oblivian@cumin1004> |
START - Cookbook sre.hosts.provision for host wikikube-worker1312.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL |
[production] |
| 04:29 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1312.eqiad.wmnet |
[production] |
| 04:29 |
<oblivian@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1312.eqiad.wmnet |
[production] |
| 04:29 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1311.eqiad.wmnet |
[production] |
| 04:27 |
<oblivian@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1311.eqiad.wmnet |
[production] |
| 04:27 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1311.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL |
[production] |
| 04:17 |
<oblivian@cumin1004> |
START - Cookbook sre.hosts.provision for host wikikube-worker1311.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL |
[production] |
| 04:16 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1311.eqiad.wmnet |
[production] |
| 04:16 |
<oblivian@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1311.eqiad.wmnet |
[production] |
| 04:16 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1310.eqiad.wmnet |
[production] |
| 04:16 |
<oblivian@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1310.eqiad.wmnet |
[production] |
| 04:15 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1310.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL |
[production] |
| 04:12 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2195.codfw.wmnet |
[production] |
| 04:12 |
<oblivian@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2195.codfw.wmnet |
[production] |
| 04:12 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2192.codfw.wmnet |
[production] |
| 04:12 |
<oblivian@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2192.codfw.wmnet |
[production] |
| 04:11 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for wikikube-worker2195.codfw.wmnet |
[production] |
| 04:11 |
<oblivian@cumin1004> |
START - Cookbook sre.hosts.remove-downtime for wikikube-worker2195.codfw.wmnet |
[production] |
| 04:11 |
<oblivian@cumin1004> |
END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for wikikube-worker2192.codfw.wmnet |
[production] |
| 04:11 |
<oblivian@cumin1004> |
START - Cookbook sre.hosts.remove-downtime for wikikube-worker2192.codfw.wmnet |
[production] |
| 02:54 |
<tstarling@deploy1003> |
Finished scap sync-world: Backport for Fix {{#invoke:}} erroring when a function returns nil before other values (T439809) ([[gerrit:1355237|1.47.0-wmf.23]], [[gerrit:1355238|1.47.0-wmf.24]]) (duration: 10m 51s) |
[production] |
| 02:49 |
<tstarling@deploy1003> |
tstarling: Continuing with deployment |
[production] |
| 02:48 |
<tstarling@deploy1003> |
tstarling: Backport for Fix {{#invoke:}} erroring when a function returns nil before other values (T439809) ([[gerrit:1355237|1.47.0-wmf.23]], [[gerrit:1355238|1.47.0-wmf.24]]) synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 02:43 |
<tstarling@deploy1003> |
Started scap sync-world: Backport for Fix {{#invoke:}} erroring when a function returns nil before other values (T439809) ([[gerrit:1355237|1.47.0-wmf.23]], [[gerrit:1355238|1.47.0-wmf.24]]) |
[production] |
| 02:39 |
<andrewbogott> |
reverted live-hacking nftables max connection throttle from 32 to 16 on gitlab1004; didn't help. T441247 |
[production] |
| 02:07 |
<mwpresync@deploy1003> |
Finished scap build-images: Publishing wmf/next image (duration: 07m 23s) |
[production] |
| 02:00 |
<mwpresync@deploy1003> |
Started scap build-images: Publishing wmf/next image |
[production] |
| 01:51 |
<andrewbogott> |
live-hacking nftables max connection throttle from 32 to 16 on gitlab1004 -- T441247 |
[production] |
| 00:00 |
<root@deploy1003> |
helmfile [staging] DONE helmfile.d/services/shellbox: sync |
[production] |
| 00:00 |
<root@deploy1003> |
helmfile [staging] START helmfile.d/services/shellbox: sync |
[production] |