|
2026-06-19
§
|
| 12:21 |
<fceratto@cumin1003> |
START - Cookbook sre.hosts.remove-downtime for db2160.codfw.wmnet |
[production] |
| 12:10 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 12:08 |
<aokoth@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on phab2002.codfw.wmnet with reason: Host Replacement |
[production] |
| 12:05 |
<urbanecm@deploy1003> |
mwscript-k8s job started: GrowthExperiments:migrateMentorStatusAway.php --wiki=viwiki # T409170 |
[production] |
| 12:04 |
<urbanecm@deploy1003> |
mwscript-k8s job started: GrowthExperiments:MigrateMentorStatusAway --wiki=viwiki # T409170 |
[production] |
| 11:33 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 11:23 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 10:38 |
<moritzm> |
imported nodejs 24.17.0-1nodesource1 to thirdparty/node24 for trixie-wikimedia |
[production] |
| 10:37 |
<moritzm> |
imported nodejs 22.23.0-1nodesource1 to thirdparty/node22 for trixie-wikimedia |
[production] |
| 10:33 |
<btullis@puppetserver1001> |
conftool action : set/pooled=no; selector: service=kubesvc,cluster=dse-k8s,dc=codfw,name=dse-k8s-wdqs2004.codfw.wmnet |
[production] |
| 10:33 |
<btullis@puppetserver1001> |
conftool action : set/pooled=no; selector: service=kubesvc,cluster=dse-k8s,dc=codfw,name=dse-k8s-wdqs2003.codfw.wmnet |
[production] |
| 10:33 |
<btullis@puppetserver1001> |
conftool action : set/pooled=no; selector: service=kubesvc,cluster=dse-k8s,dc=codfw,name=dse-k8s-wdqs2002.codfw.wmnet |
[production] |
| 10:33 |
<btullis@puppetserver1001> |
conftool action : set/pooled=no; selector: service=kubesvc,cluster=dse-k8s,dc=codfw,name=dse-k8s-wdqs2001.codfw.wmnet |
[production] |
| 10:29 |
<sergi0> |
Run `MigrateMentorStatusAway` script for all wikis in growthexperiments dblist - T409170 |
[production] |
| 10:16 |
<btullis@cumin1003> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dse-k8s-worker1020.eqiad.wmnet |
[production] |
| 10:09 |
<btullis@cumin1003> |
START - Cookbook sre.hosts.reboot-single for host dse-k8s-worker1020.eqiad.wmnet |
[production] |
| 10:04 |
<cmooney@cumin1003> |
END (PASS) - Cookbook sre.network.host-bgp (exit_code=0) for host dse-k8s-worker1024 |
[production] |
| 10:03 |
<cmooney@cumin1003> |
START - Cookbook sre.network.host-bgp for host dse-k8s-worker1024 |
[production] |
| 10:03 |
<cmooney@cumin1003> |
END (PASS) - Cookbook sre.network.host-bgp (exit_code=0) for host dse-k8s-worker1023 |
[production] |
| 10:03 |
<cmooney@cumin1003> |
START - Cookbook sre.network.host-bgp for host dse-k8s-worker1023 |
[production] |
| 10:03 |
<cmooney@cumin1003> |
END (PASS) - Cookbook sre.network.host-bgp (exit_code=0) for host dse-k8s-worker1021 |
[production] |
| 10:03 |
<cmooney@cumin1003> |
START - Cookbook sre.network.host-bgp for host dse-k8s-worker1021 |
[production] |
| 10:00 |
<cmooney@cumin1003> |
END (PASS) - Cookbook sre.network.host-bgp (exit_code=0) for host dse-k8s-worker1024 |
[production] |
| 09:59 |
<cmooney@cumin1003> |
START - Cookbook sre.network.host-bgp for host dse-k8s-worker1024 |
[production] |
| 09:58 |
<btullis@cumin1003> |
END (PASS) - Cookbook sre.network.host-bgp (exit_code=0) for host dse-k8s-worker1022 |
[production] |
| 09:57 |
<btullis@cumin1003> |
START - Cookbook sre.network.host-bgp for host dse-k8s-worker1022 |
[production] |
| 09:56 |
<cmooney@cumin1003> |
END (PASS) - Cookbook sre.network.host-bgp (exit_code=0) for host dse-k8s-worker1020 |
[production] |
| 09:54 |
<cmooney@cumin1003> |
START - Cookbook sre.network.host-bgp for host dse-k8s-worker1020 |
[production] |
| 09:43 |
<btullis@cumin1003> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dse-k8s-worker1020.eqiad.wmnet |
[production] |
| 09:39 |
<Lucas_WMDE> |
deployment-deploy04: used createAndPromote to restore User:Lucas Werkmeister (WMDE) to bureaucrat (dewiki, enwiki, metawiki, wikidatawiki) and wikidata-staff (wikidatawiki) after groups were very unhelpfully removed, presumably due to 2FA enforcement, without apparent warning or announcement of any kind |
[releng] |
| 09:36 |
<btullis@cumin1003> |
START - Cookbook sre.hosts.reboot-single for host dse-k8s-worker1020.eqiad.wmnet |
[production] |
| 07:32 |
<slyngs> |
Update IDP/SSO to CAS v7.3.7.3 |
[production] |
| 07:31 |
<slyngshede@dns1004> |
END - running authdns-update |
[production] |
| 07:30 |
<slyngshede@dns1004> |
START - running authdns-update |
[production] |
| 07:07 |
<dcausse> |
restarted php-fpm on deployment-jobrunner05 (inconsistent php state: MediaWiki\Extension\PageAssessments\HookHandler\ParserHooks::__construct(): Argument #3 ($config) must be of type MediaWiki\Config\Config, MediaWiki\Page\WikiPageFactory given) |
[deployment-prep] |
| 07:07 |
<dcausse> |
restarted php-fpm on deployment-jobrunner05 (inconsistent php state: MediaWiki\Extension\PageAssessments\HookHandler\ParserHooks::__construct(): Argument #3 ($config) must be of type MediaWiki\Config\Config, MediaWiki\Page\WikiPageFactory given) |
[releng] |
| 06:13 |
<hashar> |
Upgrading Quibble jobs to 1.18.2 (retry git connections on reset) https://gerrit.wikimedia.org/r/c/integration/config/+/1304181 # T420865 |
[releng] |
| 02:07 |
<mwpresync@deploy1003> |
Finished scap build-images: Publishing wmf/next image (duration: 06m 49s) |
[production] |
| 02:00 |
<mwpresync@deploy1003> |
Started scap build-images: Publishing wmf/next image |
[production] |
| 01:19 |
<otto@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/eventgate-analytics: sync |
[production] |
| 01:18 |
<otto@deploy1003> |
helmfile [eqiad] START helmfile.d/services/eventgate-analytics: sync |
[production] |
| 01:18 |
<otto@deploy1003> |
helmfile [codfw] DONE helmfile.d/services/eventgate-analytics: sync |
[production] |
| 01:17 |
<otto@deploy1003> |
helmfile [codfw] START helmfile.d/services/eventgate-analytics: sync |
[production] |
| 01:17 |
<otto@deploy1003> |
helmfile [staging] DONE helmfile.d/services/eventgate-analytics: sync |
[production] |
| 01:17 |
<otto@deploy1003> |
helmfile [staging] START helmfile.d/services/eventgate-analytics: sync |
[production] |
| 01:06 |
<ottomata> |
roll restart eventgate-analytics to pick up stream config change - T427787 |
[production] |