|
2025-06-20
ยง
|
| 10:11 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2219 (T396130)', diff saved to https://phabricator.wikimedia.org/P78527 and previous config saved to /var/cache/conftool/dbconfig/20250620-101116-marostegui.json |
[production] |
| 10:05 |
<oblivian@cumin1003> |
END (PASS) - Cookbook sre.deploy.hiddenparma (exit_code=0) Hiddenparma deployment to the alerting hosts with reason: "Bugfixes - oblivian@cumin1003" |
[production] |
| 10:05 |
<oblivian@cumin1003> |
END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) hiddenparma to alert[1002,2002].wikimedia.org with reason: Bugfixes - oblivian@cumin1003 |
[production] |
| 10:04 |
<oblivian@cumin1003> |
START - Cookbook sre.deploy.python-code hiddenparma to alert[1002,2002].wikimedia.org with reason: Bugfixes - oblivian@cumin1003 |
[production] |
| 10:04 |
<oblivian@cumin1003> |
START - Cookbook sre.deploy.hiddenparma Hiddenparma deployment to the alerting hosts with reason: "Bugfixes - oblivian@cumin1003" |
[production] |
| 10:04 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db2219 (T396130)', diff saved to https://phabricator.wikimedia.org/P78526 and previous config saved to /var/cache/conftool/dbconfig/20250620-100431-marostegui.json |
[production] |
| 10:04 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2219.codfw.wmnet with reason: Maintenance |
[production] |
| 10:04 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2210 (T396130)', diff saved to https://phabricator.wikimedia.org/P78525 and previous config saved to /var/cache/conftool/dbconfig/20250620-100409-marostegui.json |
[production] |
| 09:58 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply |
[production] |
| 09:57 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply |
[production] |
| 09:56 |
<jakob@deploy1003> |
helmfile [staging] DONE helmfile.d/services/wikidata-query-gui: apply |
[production] |
| 09:56 |
<jakob@deploy1003> |
helmfile [staging] START helmfile.d/services/wikidata-query-gui: apply |
[production] |
| 09:55 |
<root@cumin1002> |
DONE (PASS) - Cookbook sre.puppet.renew-cert (exit_code=0) for backup2009.codfw.wmnet: Renew puppet certificate - root@cumin1002 |
[production] |
| 09:49 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2210', diff saved to https://phabricator.wikimedia.org/P78524 and previous config saved to /var/cache/conftool/dbconfig/20250620-094901-marostegui.json |
[production] |
| 09:44 |
<jakob@deploy1003> |
helmfile [staging] DONE helmfile.d/services/wikidata-query-gui: apply |
[production] |
| 09:44 |
<jakob@deploy1003> |
helmfile [staging] START helmfile.d/services/wikidata-query-gui: apply |
[production] |
| 09:33 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2210', diff saved to https://phabricator.wikimedia.org/P78523 and previous config saved to /var/cache/conftool/dbconfig/20250620-093354-marostegui.json |
[production] |
| 09:24 |
<jynus@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on backup2009.codfw.wmnet with reason: Maintenance and reboot |
[production] |
| 09:23 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 09:18 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply |
[production] |
| 09:18 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2210 (T396130)', diff saved to https://phabricator.wikimedia.org/P78522 and previous config saved to /var/cache/conftool/dbconfig/20250620-091847-marostegui.json |
[production] |
| 09:18 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply |
[production] |
| 09:10 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db2210 (T396130)', diff saved to https://phabricator.wikimedia.org/P78521 and previous config saved to /var/cache/conftool/dbconfig/20250620-091005-marostegui.json |
[production] |
| 09:09 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2210.codfw.wmnet with reason: Maintenance |
[production] |
| 09:09 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2206 (T396130)', diff saved to https://phabricator.wikimedia.org/P78520 and previous config saved to /var/cache/conftool/dbconfig/20250620-090943-marostegui.json |
[production] |
| 08:54 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2206', diff saved to https://phabricator.wikimedia.org/P78519 and previous config saved to /var/cache/conftool/dbconfig/20250620-085435-marostegui.json |
[production] |
| 08:39 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2206', diff saved to https://phabricator.wikimedia.org/P78518 and previous config saved to /var/cache/conftool/dbconfig/20250620-083928-marostegui.json |
[production] |
| 08:24 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2206 (T396130)', diff saved to https://phabricator.wikimedia.org/P78517 and previous config saved to /var/cache/conftool/dbconfig/20250620-082420-marostegui.json |
[production] |
| 08:16 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db2206 (T396130)', diff saved to https://phabricator.wikimedia.org/P78516 and previous config saved to /var/cache/conftool/dbconfig/20250620-081638-marostegui.json |
[production] |
| 08:16 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2206.codfw.wmnet with reason: Maintenance |
[production] |
| 08:11 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2199.codfw.wmnet with reason: Maintenance |
[production] |
| 08:11 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2179 (T396130)', diff saved to https://phabricator.wikimedia.org/P78515 and previous config saved to /var/cache/conftool/dbconfig/20250620-081127-marostegui.json |
[production] |
| 07:59 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repool pc2 T378715', diff saved to https://phabricator.wikimedia.org/P78514 and previous config saved to /var/cache/conftool/dbconfig/20250620-075944-root.json |
[production] |
| 07:56 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2179', diff saved to https://phabricator.wikimedia.org/P78513 and previous config saved to /var/cache/conftool/dbconfig/20250620-075619-marostegui.json |
[production] |
| 07:41 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2179', diff saved to https://phabricator.wikimedia.org/P78512 and previous config saved to /var/cache/conftool/dbconfig/20250620-074112-marostegui.json |
[production] |
| 07:26 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2179 (T396130)', diff saved to https://phabricator.wikimedia.org/P78511 and previous config saved to /var/cache/conftool/dbconfig/20250620-072605-marostegui.json |
[production] |
| 07:20 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on es2045.codfw.wmnet with reason: Firmware downgrade pending |
[production] |
| 07:17 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db2179 (T396130)', diff saved to https://phabricator.wikimedia.org/P78510 and previous config saved to /var/cache/conftool/dbconfig/20250620-071730-marostegui.json |
[production] |
| 07:17 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2179.codfw.wmnet with reason: Maintenance |
[production] |
| 07:17 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2172 (T396130)', diff saved to https://phabricator.wikimedia.org/P78509 and previous config saved to /var/cache/conftool/dbconfig/20250620-071707-marostegui.json |
[production] |
| 07:12 |
<jmm@cumin1003> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts ganeti2024.codfw.wmnet |
[production] |
| 07:12 |
<jmm@cumin1003> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 07:12 |
<jmm@cumin1003> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: ganeti2024.codfw.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin1003" |
[production] |
| 07:11 |
<jmm@cumin1003> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: ganeti2024.codfw.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin1003" |
[production] |
| 07:02 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2172', diff saved to https://phabricator.wikimedia.org/P78508 and previous config saved to /var/cache/conftool/dbconfig/20250620-070200-marostegui.json |
[production] |
| 06:56 |
<jmm@cumin1003> |
START - Cookbook sre.dns.netbox |
[production] |
| 06:51 |
<jmm@cumin1003> |
START - Cookbook sre.hosts.decommission for hosts ganeti2024.codfw.wmnet |
[production] |
| 06:50 |
<jmm@cumin1003> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts ganeti2023.codfw.wmnet |
[production] |
| 06:50 |
<jmm@cumin1003> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 06:50 |
<jmm@cumin1003> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: ganeti2023.codfw.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin1003" |
[production] |