|
2024-03-13
§
|
| 20:24 |
<jdrewniak@deploy2002> |
jdlrobson and jdrewniak: Backport for [[gerrit:1010585|Fix Issue with localization of special page titles in exclusion logic (T359958)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 20:22 |
<jdrewniak@deploy2002> |
Started scap: Backport for [[gerrit:1010585|Fix Issue with localization of special page titles in exclusion logic (T359958)]] |
[production] |
| 19:49 |
<logmsgbot> |
@deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 19:49 |
<logmsgbot> |
@deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 19:49 |
<cstone> |
civicrm upgraded from 3cab2177 to ee785ecd |
[production] |
| 19:21 |
<andrewbogott> |
shutting down old puppet infra: tools-puppetmaster-02 and tools-puppetdb-1. These can be deleted in a week or two presuming everything remains stable. |
[tools] |
| 18:32 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1182 (re)pooling @ 100%: After recloning db1246', diff saved to https://phabricator.wikimedia.org/P58786 and previous config saved to /var/cache/conftool/dbconfig/20240313-183256-root.json |
[production] |
| 18:17 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1182 (re)pooling @ 75%: After recloning db1246', diff saved to https://phabricator.wikimedia.org/P58785 and previous config saved to /var/cache/conftool/dbconfig/20240313-181751-root.json |
[production] |
| 18:02 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1182 (re)pooling @ 50%: After recloning db1246', diff saved to https://phabricator.wikimedia.org/P58784 and previous config saved to /var/cache/conftool/dbconfig/20240313-180245-root.json |
[production] |
| 17:52 |
<ejegg> |
donorwiki upgraded from 5755ea82 to 32789f89 |
[production] |
| 17:47 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1182 (re)pooling @ 25%: After recloning db1246', diff saved to https://phabricator.wikimedia.org/P58783 and previous config saved to /var/cache/conftool/dbconfig/20240313-174740-root.json |
[production] |
| 17:44 |
<logmsgbot> |
@deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 17:43 |
<logmsgbot> |
@deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 17:35 |
<logmsgbot> |
@deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 17:34 |
<logmsgbot> |
@deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 17:32 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1182 (re)pooling @ 10%: After recloning db1246', diff saved to https://phabricator.wikimedia.org/P58782 and previous config saved to /var/cache/conftool/dbconfig/20240313-173234-root.json |
[production] |
| 17:32 |
<James_F> |
Zuul: Make 'standalone' jobs run consecutively, not concurrently |
[releng] |
| 17:26 |
<logmsgbot> |
@deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 17:26 |
<logmsgbot> |
@deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 17:17 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1182 (re)pooling @ 5%: After recloning db1246', diff saved to https://phabricator.wikimedia.org/P58781 and previous config saved to /var/cache/conftool/dbconfig/20240313-171728-root.json |
[production] |
| 17:02 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'db1182 (re)pooling @ 1%: After recloning db1246', diff saved to https://phabricator.wikimedia.org/P58779 and previous config saved to /var/cache/conftool/dbconfig/20240313-170222-root.json |
[production] |
| 17:00 |
<elukey@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'readability' for release 'main' . |
[production] |
| 16:52 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db1182.eqiad.wmnet onto db1246.eqiad.wmnet |
[production] |
| 16:48 |
<root@cumin1002> |
END (PASS) - Cookbook sre.discovery.datacenter (exit_code=0) status all services in all: None - None |
[production] |
| 16:48 |
<root@cumin1002> |
START - Cookbook sre.discovery.datacenter status all services in all: None - None |
[production] |
| 16:33 |
<eevans@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 30 days, 0:00:00 on restbase1021.eqiad.wmnet with reason: Decommissioning — T354561 |
[production] |
| 16:33 |
<eevans@cumin1002> |
START - Cookbook sre.hosts.downtime for 30 days, 0:00:00 on restbase1021.eqiad.wmnet with reason: Decommissioning — T354561 |
[production] |
| 16:32 |
<root@cumin1002> |
END (PASS) - Cookbook sre.discovery.datacenter (exit_code=0) status all services in all: None - None |
[production] |
| 16:32 |
<root@cumin1002> |
START - Cookbook sre.discovery.datacenter status all services in all: None - None |
[production] |
| 16:21 |
<wmbot~bd808@tools-sgebastion-11> |
Changed branch protections on https://gitlab.wikimedia.org/toolforge-repos/wikibugs2 to disallow push to main by all. fa4fbd54 was accidentally pushed to main rather than a feature branch. |
[tools.wikibugs] |
| 16:18 |
<wmbot~bd808@tools-sgebastion-11> |
Restared gerrit task to pick up fa4fbd54 fix for T360038 |
[tools.wikibugs] |
| 16:15 |
<aborrero@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node toolsbeta-test-k8s-control-7 from 1.23.17 to 1.27.17 (T359638) |
[toolsbeta] |
| 16:15 |
<aborrero@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-7 from 1.23.17 to 1.27.17 (T359638) |
[toolsbeta] |
| 16:14 |
<aborrero@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node toolsbeta-test-k8s-control-7 from 1.23.17 to 1.27.17 (T359638) |
[toolsbeta] |
| 16:14 |
<aborrero@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-7 from 1.23.17 to 1.27.17 (T359638) |
[toolsbeta] |
| 16:08 |
<wmbot~bd808@tools-sgebastion-11> |
Restared gerrit task to pick up 82caa471 fix for T360038 |
[tools.wikibugs] |
| 16:07 |
<logmsgbot> |
@deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 16:07 |
<logmsgbot> |
@deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 16:01 |
<hashar@deploy2002> |
Synchronized php: group1 wikis to 1.42.0-wmf.22 refs T354440 (duration: 11m 46s) |
[production] |
| 15:49 |
<aborrero@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=0) for cluster toolsbeta upgrade from 1.23 to 1.24 (T359638) |
[toolsbeta] |
| 15:49 |
<hashar@deploy2002> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.42.0-wmf.22 refs T354440 |
[production] |
| 15:48 |
<aborrero@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster toolsbeta upgrade from 1.23 to 1.24 (T359638) |
[toolsbeta] |
| 15:32 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy2002 Finished scap: Backport for [[gerrit:1010581|WikiModule: Fix pages merging (T360014)]] (duration: 13m 27s) |
[production] |
| 15:27 |
<marostegui@cumin1002> |
START - Cookbook sre.mysql.clone of db1182.eqiad.wmnet onto db1246.eqiad.wmnet |
[production] |
| 15:25 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1246.eqiad.wmnet with OS bookworm |
[production] |
| 15:25 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - marostegui@cumin1002" |
[production] |
| 15:24 |
<marostegui@cumin1002> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - marostegui@cumin1002" |
[production] |
| 15:22 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy2002 lucaswerkmeister-wmde: Continuing with sync |
[production] |
| 15:21 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy2002 lucaswerkmeister-wmde: Backport for [[gerrit:1010581|WikiModule: Fix pages merging (T360014)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 15:19 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy2002 Started scap: Backport for [[gerrit:1010581|WikiModule: Fix pages merging (T360014)]] |
[production] |