|
2024-07-05
ยง
|
| 17:00 |
<logmsgbot> |
andrewtavis-wmde@deploy1002 Started deploy [airflow-dags/wmde@73c6618]: (no justification provided) |
[production] |
| 17:00 |
<wmbot~bd808@tools-bastion-12> |
Replaced tool-bridgebot/znc container with tool-containers/bnc container (T366970) |
[tools.bridgebot] |
| 15:49 |
<arturo> |
deleted a bunch of leaked VMs -- timeout waiting for A record |
[admin-monitoring] |
| 13:40 |
<hashar@deploy1002> |
Finished deploy [integration/docroot@18c8279]: Add AQS documentation to landing page - T368484 (duration: 00m 06s) |
[production] |
| 13:40 |
<hashar@deploy1002> |
Started deploy [integration/docroot@18c8279]: Add AQS documentation to landing page - T368484 |
[production] |
| 12:57 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1246.eqiad.wmnet with reason: Long schema change |
[production] |
| 12:57 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1246.eqiad.wmnet with reason: Long schema change |
[production] |
| 12:52 |
<aborrero@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno |
[tools] |
| 12:51 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db2157 (T367856)', diff saved to https://phabricator.wikimedia.org/P65869 and previous config saved to /var/cache/conftool/dbconfig/20240705-125152-marostegui.json |
[production] |
| 12:51 |
<aborrero@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno |
[tools] |
| 12:51 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2157.codfw.wmnet with reason: Maintenance |
[production] |
| 12:51 |
<aborrero@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno |
[toolsbeta] |
| 12:51 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2157.codfw.wmnet with reason: Maintenance |
[production] |
| 12:51 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2128 (T367856)', diff saved to https://phabricator.wikimedia.org/P65868 and previous config saved to /var/cache/conftool/dbconfig/20240705-125130-marostegui.json |
[production] |
| 12:51 |
<aborrero@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno |
[toolsbeta] |
| 12:36 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2128', diff saved to https://phabricator.wikimedia.org/P65867 and previous config saved to /var/cache/conftool/dbconfig/20240705-123623-marostegui.json |
[production] |
| 12:34 |
<aborrero@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno |
[tools] |
| 12:34 |
<aborrero@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno |
[tools] |
| 12:33 |
<aborrero@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno |
[toolsbeta] |
| 12:33 |
<aborrero@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno |
[toolsbeta] |
| 12:29 |
<wmbot~arturo@nostromo> |
END (PASS) - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry (exit_code=0) |
[tools] |
| 12:29 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/bitnami-kubectl:1.26.4 |
[tools] |
| 12:29 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-reports-controller:v1.10.7 |
[tools] |
| 12:28 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-cleanup-controller:v1.10.7 |
[tools] |
| 12:28 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-background-controller:v1.10.7 |
[tools] |
| 12:28 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyvernopre:v1.10.7 |
[tools] |
| 12:28 |
<wmbot~lucaswerkmeister@tools-bastion-13> |
deployed 1013a7234d (l10n updates: ar, de, uk) |
[tools.lexeme-forms] |
| 12:28 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyverno:v1.10.7 |
[tools] |
| 12:27 |
<wmbot~arturo@nostromo> |
START - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry |
[tools] |
| 12:27 |
<wmbot~arturo@nostromo> |
END (FAIL) - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry (exit_code=99) |
[tools] |
| 12:26 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-reports-controller:v1.10.7 |
[tools] |
| 12:26 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-cleanup-controller:v1.10.7 |
[tools] |
| 12:26 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-background-controller:v1.10.7 |
[tools] |
| 12:26 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyvernopre:v1.10.7 |
[tools] |
| 12:26 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyverno:v1.10.7 |
[tools] |
| 12:26 |
<wmbot~arturo@nostromo> |
START - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry |
[tools] |
| 12:23 |
<sstefanova@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) |
[tools] |
| 12:23 |
<sstefanova@cloudcumin1001> |
Updating container image docker-registry.tools.wmflabs.org/kube-state-metrics:v2.7.0 |
[tools] |
| 12:23 |
<sstefanova@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry |
[tools] |
| 12:21 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2128', diff saved to https://phabricator.wikimedia.org/P65866 and previous config saved to /var/cache/conftool/dbconfig/20240705-122115-marostegui.json |
[production] |
| 12:19 |
<btullis@deploy1002> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 12:19 |
<btullis@deploy1002> |
helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 12:06 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2128 (T367856)', diff saved to https://phabricator.wikimedia.org/P65865 and previous config saved to /var/cache/conftool/dbconfig/20240705-120608-marostegui.json |
[production] |
| 11:57 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'db1231 (re)pooling @ 100%: Maint over', diff saved to https://phabricator.wikimedia.org/P65864 and previous config saved to /var/cache/conftool/dbconfig/20240705-115703-ladsgroup.json |
[production] |
| 11:53 |
<dcausse> |
T369149: re-indexed wikidata P12861 (cirrus_rerender.rerender --wiki wikidatawiki allpages --namespace 120 --from-title P12861 --to-title P12861) |
[production] |
| 11:41 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'db1231 (re)pooling @ 75%: Maint over', diff saved to https://phabricator.wikimedia.org/P65863 and previous config saved to /var/cache/conftool/dbconfig/20240705-114157-ladsgroup.json |
[production] |
| 11:30 |
<cgoubert@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on kubernetes1051.eqiad.wmnet with reason: Hardware issue |
[production] |
| 11:29 |
<cgoubert@cumin1002> |
START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on kubernetes1051.eqiad.wmnet with reason: Hardware issue |
[production] |
| 11:29 |
<wmbot~arturo@nostromo> |
END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) copy image from bitnami/kubectl:1.26.4 to docker-registry.tools.wmflabs.org/bitnami-kubectl:1.26.4 |
[tools] |
| 11:28 |
<wmbot~arturo@nostromo> |
Updating container image docker-registry.tools.wmflabs.org/bitnami-kubectl:1.26.4 |
[tools] |