|
2025-06-10
ยง
|
| 09:17 |
<jynus@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2197.codfw.wmnet,dbprov2003.codfw.wmnet with reason: Downtime hosts for MariaDB 10.11 upgrade |
[production] |
| 09:15 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 09:10 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Depooling db1170 (T395241)', diff saved to https://phabricator.wikimedia.org/P77441 and previous config saved to /var/cache/conftool/dbconfig/20250610-091040-fceratto.json |
[production] |
| 09:10 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1170.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:10 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1158 (T395241)', diff saved to https://phabricator.wikimedia.org/P77440 and previous config saved to /var/cache/conftool/dbconfig/20250610-091016-fceratto.json |
[production] |
| 09:08 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:07 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1245.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:06 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1216.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:06 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1210 (T396130)', diff saved to https://phabricator.wikimedia.org/P77439 and previous config saved to /var/cache/conftool/dbconfig/20250610-090635-marostegui.json |
[production] |
| 08:55 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1158', diff saved to https://phabricator.wikimedia.org/P77438 and previous config saved to /var/cache/conftool/dbconfig/20250610-085508-fceratto.json |
[production] |
| 08:54 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 08:53 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 08:53 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 08:51 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1210', diff saved to https://phabricator.wikimedia.org/P77437 and previous config saved to /var/cache/conftool/dbconfig/20250610-085128-marostegui.json |
[production] |
| 08:40 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 08:40 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1158', diff saved to https://phabricator.wikimedia.org/P77436 and previous config saved to /var/cache/conftool/dbconfig/20250610-084002-fceratto.json |
[production] |
| 08:36 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1210', diff saved to https://phabricator.wikimedia.org/P77435 and previous config saved to /var/cache/conftool/dbconfig/20250610-083622-marostegui.json |
[production] |
| 08:24 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1158 (T395241)', diff saved to https://phabricator.wikimedia.org/P77434 and previous config saved to /var/cache/conftool/dbconfig/20250610-082454-fceratto.json |
[production] |
| 08:21 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1210 (T396130)', diff saved to https://phabricator.wikimedia.org/P77433 and previous config saved to /var/cache/conftool/dbconfig/20250610-082114-marostegui.json |
[production] |
| 08:18 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1210 (T396130)', diff saved to https://phabricator.wikimedia.org/P77432 and previous config saved to /var/cache/conftool/dbconfig/20250610-081817-marostegui.json |
[production] |
| 08:18 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1210.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:17 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1200 (T396130)', diff saved to https://phabricator.wikimedia.org/P77431 and previous config saved to /var/cache/conftool/dbconfig/20250610-081756-marostegui.json |
[production] |
| 08:16 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Depooling db1158 (T395241)', diff saved to https://phabricator.wikimedia.org/P77430 and previous config saved to /var/cache/conftool/dbconfig/20250610-081647-fceratto.json |
[production] |
| 08:16 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1014,1018].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:16 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1158.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:06 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 32098 |
[production] |
| 08:05 |
<ayounsi@cumin1002> |
START - Cookbook sre.network.peering with action 'configure' for AS: 32098 |
[production] |
| 08:02 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1200', diff saved to https://phabricator.wikimedia.org/P77429 and previous config saved to /var/cache/conftool/dbconfig/20250610-080248-marostegui.json |
[production] |
| 08:01 |
<jynus> |
deploying grants for zuul backups @ m1 T394844 |
[production] |
| 07:47 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1200', diff saved to https://phabricator.wikimedia.org/P77428 and previous config saved to /var/cache/conftool/dbconfig/20250610-074742-marostegui.json |
[production] |
| 07:36 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'es2031 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P77427 and previous config saved to /var/cache/conftool/dbconfig/20250610-073631-root.json |
[production] |
| 07:32 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1200 (T396130)', diff saved to https://phabricator.wikimedia.org/P77426 and previous config saved to /var/cache/conftool/dbconfig/20250610-073234-marostegui.json |
[production] |
| 07:30 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1200 (T396130)', diff saved to https://phabricator.wikimedia.org/P77425 and previous config saved to /var/cache/conftool/dbconfig/20250610-073003-marostegui.json |
[production] |
| 07:29 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1200.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:29 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1185 (T396130)', diff saved to https://phabricator.wikimedia.org/P77424 and previous config saved to /var/cache/conftool/dbconfig/20250610-072941-marostegui.json |
[production] |
| 07:28 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 8849 |
[production] |
| 07:28 |
<ayounsi@cumin1002> |
START - Cookbook sre.network.peering with action 'email' for AS: 8849 |
[production] |
| 07:26 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 28173 |
[production] |
| 07:26 |
<ayounsi@cumin1002> |
START - Cookbook sre.network.peering with action 'email' for AS: 28173 |
[production] |
| 07:25 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 60427 |
[production] |
| 07:25 |
<ayounsi@cumin1002> |
START - Cookbook sre.network.peering with action 'email' for AS: 60427 |
[production] |
| 07:24 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 10310 |
[production] |
| 07:21 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'es2031 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P77423 and previous config saved to /var/cache/conftool/dbconfig/20250610-072125-root.json |
[production] |
| 07:21 |
<ayounsi@cumin1002> |
START - Cookbook sre.network.peering with action 'email' for AS: 10310 |
[production] |
| 07:14 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1185', diff saved to https://phabricator.wikimedia.org/P77422 and previous config saved to /var/cache/conftool/dbconfig/20250610-071434-marostegui.json |
[production] |
| 07:14 |
<jmm@cumin1003> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host install7002.wikimedia.org |
[production] |
| 07:14 |
<jmm@cumin1003> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host install7002.wikimedia.org with OS bookworm |
[production] |
| 07:06 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'es2031 (re)pooling @ 60%: Repooling', diff saved to https://phabricator.wikimedia.org/P77421 and previous config saved to /var/cache/conftool/dbconfig/20250610-070620-root.json |
[production] |
| 07:02 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'es2030 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P77420 and previous config saved to /var/cache/conftool/dbconfig/20250610-070240-root.json |
[production] |
| 06:59 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1185', diff saved to https://phabricator.wikimedia.org/P77419 and previous config saved to /var/cache/conftool/dbconfig/20250610-065927-marostegui.json |
[production] |