|
2024-07-05
ยง
|
| 12:51 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2128 (T367856)', diff saved to https://phabricator.wikimedia.org/P65868 and previous config saved to /var/cache/conftool/dbconfig/20240705-125130-marostegui.json |
[production] |
| 12:51 |
<aborrero@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno |
[toolsbeta] |
| 12:36 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2128', diff saved to https://phabricator.wikimedia.org/P65867 and previous config saved to /var/cache/conftool/dbconfig/20240705-123623-marostegui.json |
[production] |
| 12:34 |
<aborrero@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno |
[tools] |
| 12:34 |
<aborrero@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno |
[tools] |
| 12:33 |
<aborrero@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno |
[toolsbeta] |
| 12:33 |
<aborrero@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno |
[toolsbeta] |
| 12:29 |
<wmbot~arturo@nostromo> |
END (PASS) - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry (exit_code=0) |
[tools] |
| 12:29 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/bitnami-kubectl:1.26.4 |
[tools] |
| 12:29 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-reports-controller:v1.10.7 |
[tools] |
| 12:28 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-cleanup-controller:v1.10.7 |
[tools] |
| 12:28 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-background-controller:v1.10.7 |
[tools] |
| 12:28 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyvernopre:v1.10.7 |
[tools] |
| 12:28 |
<wmbot~lucaswerkmeister@tools-bastion-13> |
deployed 1013a7234d (l10n updates: ar, de, uk) |
[tools.lexeme-forms] |
| 12:28 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyverno:v1.10.7 |
[tools] |
| 12:27 |
<wmbot~arturo@nostromo> |
START - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry |
[tools] |
| 12:27 |
<wmbot~arturo@nostromo> |
END (FAIL) - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry (exit_code=99) |
[tools] |
| 12:26 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-reports-controller:v1.10.7 |
[tools] |
| 12:26 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-cleanup-controller:v1.10.7 |
[tools] |
| 12:26 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-background-controller:v1.10.7 |
[tools] |
| 12:26 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyvernopre:v1.10.7 |
[tools] |
| 12:26 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyverno:v1.10.7 |
[tools] |
| 12:26 |
<wmbot~arturo@nostromo> |
START - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry |
[tools] |
| 12:23 |
<sstefanova@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) |
[tools] |
| 12:23 |
<sstefanova@cloudcumin1001> |
Updating container image docker-registry.tools.wmflabs.org/kube-state-metrics:v2.7.0 |
[tools] |
| 12:23 |
<sstefanova@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry |
[tools] |
| 12:21 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2128', diff saved to https://phabricator.wikimedia.org/P65866 and previous config saved to /var/cache/conftool/dbconfig/20240705-122115-marostegui.json |
[production] |
| 12:19 |
<btullis@deploy1002> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 12:19 |
<btullis@deploy1002> |
helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 12:06 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2128 (T367856)', diff saved to https://phabricator.wikimedia.org/P65865 and previous config saved to /var/cache/conftool/dbconfig/20240705-120608-marostegui.json |
[production] |
| 11:57 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'db1231 (re)pooling @ 100%: Maint over', diff saved to https://phabricator.wikimedia.org/P65864 and previous config saved to /var/cache/conftool/dbconfig/20240705-115703-ladsgroup.json |
[production] |
| 11:53 |
<dcausse> |
T369149: re-indexed wikidata P12861 (cirrus_rerender.rerender --wiki wikidatawiki allpages --namespace 120 --from-title P12861 --to-title P12861) |
[production] |
| 11:41 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'db1231 (re)pooling @ 75%: Maint over', diff saved to https://phabricator.wikimedia.org/P65863 and previous config saved to /var/cache/conftool/dbconfig/20240705-114157-ladsgroup.json |
[production] |
| 11:30 |
<cgoubert@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on kubernetes1051.eqiad.wmnet with reason: Hardware issue |
[production] |
| 11:29 |
<cgoubert@cumin1002> |
START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on kubernetes1051.eqiad.wmnet with reason: Hardware issue |
[production] |
| 11:29 |
<wmbot~arturo@nostromo> |
END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) copy image from bitnami/kubectl:1.26.4 to docker-registry.tools.wmflabs.org/bitnami-kubectl:1.26.4 |
[tools] |
| 11:28 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/bitnami-kubectl:1.26.4 |
[tools] |
| 11:28 |
<wmbot~arturo@nostromo> |
START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry copy image from bitnami/kubectl:1.26.4 to docker-registry.tools.wmflabs.org/bitnami-kubectl:1.26.4 |
[tools] |
| 11:26 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'db1231 (re)pooling @ 25%: Maint over', diff saved to https://phabricator.wikimedia.org/P65862 and previous config saved to /var/cache/conftool/dbconfig/20240705-112652-ladsgroup.json |
[production] |
| 11:13 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db2207 (T352010)', diff saved to https://phabricator.wikimedia.org/P65861 and previous config saved to /var/cache/conftool/dbconfig/20240705-111322-ladsgroup.json |
[production] |
| 11:13 |
<ladsgroup@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2207.codfw.wmnet with reason: Maintenance |
[production] |
| 11:13 |
<ladsgroup@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2207.codfw.wmnet with reason: Maintenance |
[production] |
| 11:11 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'db1231 (re)pooling @ 10%: Maint over', diff saved to https://phabricator.wikimedia.org/P65860 and previous config saved to /var/cache/conftool/dbconfig/20240705-111146-ladsgroup.json |
[production] |
| 10:52 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance |
[production] |
| 10:52 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance |
[production] |
| 10:41 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 Finished scap: Backport for [[gerrit:1052269|Define custom search-index-data-formatter-callback (T369149)]], [[gerrit:1052270|Try looking up search index data formatters by data type (T369149)]] (duration: 21m 22s) |
[production] |
| 10:36 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 lucaswerkmeister-wmde: Continuing with sync |
[production] |
| 10:33 |
<arturo> |
aborrero@cloudcephmon1001:~$ sudo ceph osd unset norebalance |
[admin] |
| 10:31 |
<arturo> |
aborrero@cloudcephmon1001:~$ sudo ceph osd unset noin |
[admin] |
| 10:22 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 lucaswerkmeister-wmde: Backport for [[gerrit:1052269|Define custom search-index-data-formatter-callback (T369149)]], [[gerrit:1052270|Try looking up search index data formatters by data type (T369149)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |