|
2025-06-10
ยง
|
| 09:36 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 09:35 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1170', diff saved to https://phabricator.wikimedia.org/P77444 and previous config saved to /var/cache/conftool/dbconfig/20250610-093518-fceratto.json |
[production] |
| 09:33 |
<jmm@cumin1003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2038.codfw.wmnet |
[production] |
| 09:32 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db2148 (T396130)', diff saved to https://phabricator.wikimedia.org/P77443 and previous config saved to /var/cache/conftool/dbconfig/20250610-093252-marostegui.json |
[production] |
| 09:32 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2148.codfw.wmnet with reason: Maintenance |
[production] |
| 09:31 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1230.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:31 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 09:27 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 09:26 |
<fnegri@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on clouddb1019.eqiad.wmnet with reason: Upgrading clouddbs T394372 |
[production] |
| 09:26 |
<jynus> |
upgrade db2197 to MariaDB 10.11 T394487 |
[production] |
| 09:24 |
<fnegri@cumin1003> |
conftool action : set/pooled=no; selector: name=clouddb1019.eqiad.wmnet |
[production] |
| 09:20 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1170 (T395241)', diff saved to https://phabricator.wikimedia.org/P77442 and previous config saved to /var/cache/conftool/dbconfig/20250610-092011-fceratto.json |
[production] |
| 09:17 |
<jynus@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2197.codfw.wmnet,dbprov2003.codfw.wmnet with reason: Downtime hosts for MariaDB 10.11 upgrade |
[production] |
| 09:15 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 09:11 |
<taavi> |
delete old proxy-03,04 instances T379175 |
[project-proxy] |
| 09:10 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Depooling db1170 (T395241)', diff saved to https://phabricator.wikimedia.org/P77441 and previous config saved to /var/cache/conftool/dbconfig/20250610-091040-fceratto.json |
[production] |
| 09:10 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1170.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:10 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1158 (T395241)', diff saved to https://phabricator.wikimedia.org/P77440 and previous config saved to /var/cache/conftool/dbconfig/20250610-091016-fceratto.json |
[production] |
| 09:08 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:07 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1245.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:06 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1216.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:06 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1210 (T396130)', diff saved to https://phabricator.wikimedia.org/P77439 and previous config saved to /var/cache/conftool/dbconfig/20250610-090635-marostegui.json |
[production] |
| 08:55 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1158', diff saved to https://phabricator.wikimedia.org/P77438 and previous config saved to /var/cache/conftool/dbconfig/20250610-085508-fceratto.json |
[production] |
| 08:54 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 08:53 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 08:53 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 08:51 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1210', diff saved to https://phabricator.wikimedia.org/P77437 and previous config saved to /var/cache/conftool/dbconfig/20250610-085128-marostegui.json |
[production] |
| 08:40 |
<fceratto@deploy1003> |
helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . |
[production] |
| 08:40 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1158', diff saved to https://phabricator.wikimedia.org/P77436 and previous config saved to /var/cache/conftool/dbconfig/20250610-084002-fceratto.json |
[production] |
| 08:36 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1210', diff saved to https://phabricator.wikimedia.org/P77435 and previous config saved to /var/cache/conftool/dbconfig/20250610-083622-marostegui.json |
[production] |
| 08:24 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1158 (T395241)', diff saved to https://phabricator.wikimedia.org/P77434 and previous config saved to /var/cache/conftool/dbconfig/20250610-082454-fceratto.json |
[production] |
| 08:21 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1210 (T396130)', diff saved to https://phabricator.wikimedia.org/P77433 and previous config saved to /var/cache/conftool/dbconfig/20250610-082114-marostegui.json |
[production] |
| 08:18 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1210 (T396130)', diff saved to https://phabricator.wikimedia.org/P77432 and previous config saved to /var/cache/conftool/dbconfig/20250610-081817-marostegui.json |
[production] |
| 08:18 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1210.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:17 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1200 (T396130)', diff saved to https://phabricator.wikimedia.org/P77431 and previous config saved to /var/cache/conftool/dbconfig/20250610-081756-marostegui.json |
[production] |
| 08:16 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Depooling db1158 (T395241)', diff saved to https://phabricator.wikimedia.org/P77430 and previous config saved to /var/cache/conftool/dbconfig/20250610-081647-fceratto.json |
[production] |
| 08:16 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1014,1018].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:16 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1158.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:06 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 32098 |
[production] |
| 08:05 |
<taavi> |
deploy https://gitlab.wikimedia.org/toolforge-repos/fourohfour/-/merge_requests/13 |
[tools.fourohfour] |
| 08:05 |
<ayounsi@cumin1002> |
START - Cookbook sre.network.peering with action 'configure' for AS: 32098 |
[production] |
| 08:02 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1200', diff saved to https://phabricator.wikimedia.org/P77429 and previous config saved to /var/cache/conftool/dbconfig/20250610-080248-marostegui.json |
[production] |
| 08:01 |
<jynus> |
deploying grants for zuul backups @ m1 T394844 |
[production] |
| 07:47 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1200', diff saved to https://phabricator.wikimedia.org/P77428 and previous config saved to /var/cache/conftool/dbconfig/20250610-074742-marostegui.json |
[production] |
| 07:36 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'es2031 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P77427 and previous config saved to /var/cache/conftool/dbconfig/20250610-073631-root.json |
[production] |
| 07:32 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1200 (T396130)', diff saved to https://phabricator.wikimedia.org/P77426 and previous config saved to /var/cache/conftool/dbconfig/20250610-073234-marostegui.json |
[production] |
| 07:30 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1200 (T396130)', diff saved to https://phabricator.wikimedia.org/P77425 and previous config saved to /var/cache/conftool/dbconfig/20250610-073003-marostegui.json |
[production] |
| 07:29 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1200.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:29 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1185 (T396130)', diff saved to https://phabricator.wikimedia.org/P77424 and previous config saved to /var/cache/conftool/dbconfig/20250610-072941-marostegui.json |
[production] |
| 07:28 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 8849 |
[production] |