|
2026-10-10
§
|
| 15:49 |
<wmbot~lucaswerkmeister@tools-bastion-15> |
deployed 1461047..840114a: Move OAuth access token to separate database table (including db schema update) |
[tools.quickcategories] |
| 14:32 |
<lucaswerkmeister> |
[cont.] 13283, 13301, 13434, 17412, 17445, 17477, 18154);" |
[tools.quickcategories] |
| 14:32 |
<lucaswerkmeister> |
sql tools "UPDATE s53976__quickcategories.background SET background_auth=NULL, background_stopped_utc_timestamp=$(date +%s), background_stopped_localuser=1 WHERE background_batch IN (9021, 9023, 9026, 9031, 9039, 9056, 9339, 9337, 9355, 9358, 9362, 9379, 9673, 9883, 9885, 9886, 9993, 11111, 13219, 13229, 13230, 13271, |
[tools.quickcategories] |
| 14:31 |
<lucaswerkmeister> |
sql tools "UPDATE s53976__quickcategories.background SET background_auth=NULL, background_stopped_utc_timestamp=$(date +%s), background_stopped_localuser=5 WHERE background_batch IN (7824, 13458, 13465, 13733, 13734, 13735, 13737);" |
[tools.quickcategories] |
| 14:31 |
<lucaswerkmeister> |
sql tools "UPDATE s53976__quickcategories.background SET background_auth=NULL, background_stopped_utc_timestamp=$(date +%s), background_stopped_localuser=1 WHERE background_batch IN (9926, 12139, 12882);" |
[tools.quickcategories] |
| 14:31 |
<lucaswerkmeister> |
sql tools "UPDATE s53976__quickcategories.background SET background_auth=NULL, background_stopped_utc_timestamp=$(date +%s), background_stopped_localuser=5 WHERE background_batch IN (11648, 12156);" |
[tools.quickcategories] |
| 14:31 |
<lucaswerkmeister> |
cleaning up some stale background runs in preparation for an upcoming migration to OAuth 2 (which will require OAuth tokens to be stored differently than they are now, with a uniqueness constraint that the stale rows violate) |
[tools.quickcategories] |
| 12:00 |
<root@deploy1003> |
helmfile [staging] DONE helmfile.d/services/shellbox-timeline: sync |
[production] |
| 12:00 |
<root@deploy1003> |
helmfile [staging] START helmfile.d/services/shellbox-timeline: sync |
[production] |
| 02:07 |
<mwpresync@deploy1003> |
Finished scap build-images: Publishing wmf/next image (duration: 07m 13s) |
[production] |
| 02:00 |
<mwpresync@deploy1003> |
Started scap build-images: Publishing wmf/next image |
[production] |
| 00:00 |
<root@deploy1003> |
helmfile [staging] DONE helmfile.d/services/shellbox-timeline: sync |
[production] |
| 00:00 |
<root@deploy1003> |
helmfile [staging] START helmfile.d/services/shellbox-timeline: sync |
[production] |
| 00:00 |
<root@deploy1003> |
helmfile [staging] DONE helmfile.d/services/shellbox: sync |
[production] |
| 00:00 |
<root@deploy1003> |
helmfile [staging] START helmfile.d/services/shellbox: sync |
[production] |
|
2026-10-09
§
|
| 18:54 |
<cscott@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1354042|Undeploy Parsoid readviews on frwikisource (T440695)]] (duration: 09m 58s) |
[production] |
| 18:49 |
<cscott@deploy1003> |
cscott, ssastry: Continuing with deployment |
[production] |
| 18:48 |
<cscott@deploy1003> |
cscott, ssastry: Backport for [[gerrit:1354042|Undeploy Parsoid readviews on frwikisource (T440695)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 18:44 |
<cscott@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1354042|Undeploy Parsoid readviews on frwikisource (T440695)]] |
[production] |
| 18:10 |
<andrew@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 18:10 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:38 |
<Daimona> |
Run database query from T440557#12411029 in x1.wikishared |
[production] |
| 17:37 |
<andrew@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:37 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:37 |
<andrew@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:37 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:37 |
<andrew@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:37 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:35 |
<andrew@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:35 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:33 |
<andrew@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:33 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:32 |
<andrew@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:32 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:32 |
<andrew@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:32 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:31 |
<andrew@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:31 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) |
[tools] |
| 17:30 |
<andrew@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.1.3 (T414595) |
[tools] |
| 17:30 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.1.3 (T414595) |
[tools] |
| 16:47 |
<cmooney@cumin1004> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:30:00 on netbox1003.eqiad.wmnet with reason: Restoring DB from backup on netboxdb1003 |
[production] |
| 16:05 |
<cmooney@cumin1004> |
END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox |
[production] |
| 16:01 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.vps.delete_project (exit_code=0) for project baglama2 in eqiad1 (T440700) |
[baglama2] |
| 15:57 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.vps.delete_project for project baglama2 in eqiad1 (T440700) |
[baglama2] |
| 15:50 |
<cmooney@cumin1004> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 15:50 |
<cmooney@cumin1004> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: merge new dns entries for eqiad ab links - cmooney@cumin1004" |
[production] |
| 15:50 |
<cmooney@cumin1004> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: merge new dns entries for eqiad ab links - cmooney@cumin1004" |
[production] |
| 15:46 |
<cmooney@cumin1004> |
START - Cookbook sre.dns.netbox |
[production] |
| 15:44 |
<cmooney@cumin1004> |
START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox |
[production] |
| 15:44 |
<cmooney@cumin1004> |
END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox-canary |
[production] |