|
2025-06-20
§
|
| 07:26 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2179 (T396130)', diff saved to https://phabricator.wikimedia.org/P78511 and previous config saved to /var/cache/conftool/dbconfig/20250620-072605-marostegui.json |
[production] |
| 07:20 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on es2045.codfw.wmnet with reason: Firmware downgrade pending |
[production] |
| 07:17 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db2179 (T396130)', diff saved to https://phabricator.wikimedia.org/P78510 and previous config saved to /var/cache/conftool/dbconfig/20250620-071730-marostegui.json |
[production] |
| 07:17 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2179.codfw.wmnet with reason: Maintenance |
[production] |
| 07:17 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2172 (T396130)', diff saved to https://phabricator.wikimedia.org/P78509 and previous config saved to /var/cache/conftool/dbconfig/20250620-071707-marostegui.json |
[production] |
| 07:12 |
<jmm@cumin1003> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts ganeti2024.codfw.wmnet |
[production] |
| 07:12 |
<jmm@cumin1003> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 07:12 |
<jmm@cumin1003> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: ganeti2024.codfw.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin1003" |
[production] |
| 07:11 |
<jmm@cumin1003> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: ganeti2024.codfw.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin1003" |
[production] |
| 07:02 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2172', diff saved to https://phabricator.wikimedia.org/P78508 and previous config saved to /var/cache/conftool/dbconfig/20250620-070200-marostegui.json |
[production] |
| 06:56 |
<jmm@cumin1003> |
START - Cookbook sre.dns.netbox |
[production] |
| 06:51 |
<jmm@cumin1003> |
START - Cookbook sre.hosts.decommission for hosts ganeti2024.codfw.wmnet |
[production] |
| 06:50 |
<jmm@cumin1003> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts ganeti2023.codfw.wmnet |
[production] |
| 06:50 |
<jmm@cumin1003> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 06:50 |
<jmm@cumin1003> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: ganeti2023.codfw.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin1003" |
[production] |
| 06:48 |
<jmm@cumin1003> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: ganeti2023.codfw.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin1003" |
[production] |
| 06:46 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2172', diff saved to https://phabricator.wikimedia.org/P78507 and previous config saved to /var/cache/conftool/dbconfig/20250620-064652-marostegui.json |
[production] |
| 06:44 |
<jmm@cumin1003> |
START - Cookbook sre.dns.netbox |
[production] |
| 06:40 |
<jmm@cumin1003> |
START - Cookbook sre.hosts.decommission for hosts ganeti2023.codfw.wmnet |
[production] |
| 06:31 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2172 (T396130)', diff saved to https://phabricator.wikimedia.org/P78506 and previous config saved to /var/cache/conftool/dbconfig/20250620-063145-marostegui.json |
[production] |
| 06:23 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db2172 (T396130)', diff saved to https://phabricator.wikimedia.org/P78504 and previous config saved to /var/cache/conftool/dbconfig/20250620-062307-marostegui.json |
[production] |
| 06:23 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2172.codfw.wmnet with reason: Maintenance |
[production] |
| 06:17 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2155.codfw.wmnet with reason: Maintenance |
[production] |
| 06:17 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2147 (T396130)', diff saved to https://phabricator.wikimedia.org/P78503 and previous config saved to /var/cache/conftool/dbconfig/20250620-061659-marostegui.json |
[production] |
| 06:02 |
<jmm@cumin1003> |
END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of ml-staging-etcd2002.codfw.wmnet to plain |
[production] |
| 06:02 |
<jmm@cumin1003> |
START - Cookbook sre.ganeti.changedisk for changing disk type of ml-staging-etcd2002.codfw.wmnet to plain |
[production] |
| 06:01 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2147', diff saved to https://phabricator.wikimedia.org/P78502 and previous config saved to /var/cache/conftool/dbconfig/20250620-060151-marostegui.json |
[production] |
| 05:46 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2147', diff saved to https://phabricator.wikimedia.org/P78501 and previous config saved to /var/cache/conftool/dbconfig/20250620-054644-marostegui.json |
[production] |
| 05:31 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2147 (T396130)', diff saved to https://phabricator.wikimedia.org/P78500 and previous config saved to /var/cache/conftool/dbconfig/20250620-053137-marostegui.json |
[production] |
| 05:23 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db2147 (T396130)', diff saved to https://phabricator.wikimedia.org/P78499 and previous config saved to /var/cache/conftool/dbconfig/20250620-052300-marostegui.json |
[production] |
| 05:22 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2147.codfw.wmnet with reason: Maintenance |
[production] |
| 05:21 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 15 hosts with reason: Upgrade |
[production] |
| 05:18 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1236.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:10 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on pc1012.eqiad.wmnet with reason: Maintenance |
[production] |
|
2025-06-19
§
|
| 23:40 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:40 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1253 (T396130)', diff saved to https://phabricator.wikimedia.org/P78498 and previous config saved to /var/cache/conftool/dbconfig/20250619-234009-marostegui.json |
[production] |
| 23:25 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1253', diff saved to https://phabricator.wikimedia.org/P78497 and previous config saved to /var/cache/conftool/dbconfig/20250619-232502-marostegui.json |
[production] |
| 23:09 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1253', diff saved to https://phabricator.wikimedia.org/P78496 and previous config saved to /var/cache/conftool/dbconfig/20250619-230955-marostegui.json |
[production] |
| 22:54 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1253 (T396130)', diff saved to https://phabricator.wikimedia.org/P78495 and previous config saved to /var/cache/conftool/dbconfig/20250619-225448-marostegui.json |
[production] |
| 22:49 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1253 (T396130)', diff saved to https://phabricator.wikimedia.org/P78494 and previous config saved to /var/cache/conftool/dbconfig/20250619-224901-marostegui.json |
[production] |
| 22:48 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1253.eqiad.wmnet with reason: Maintenance |
[production] |
| 22:48 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1227 (T396130)', diff saved to https://phabricator.wikimedia.org/P78493 and previous config saved to /var/cache/conftool/dbconfig/20250619-224839-marostegui.json |
[production] |
| 22:33 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1227', diff saved to https://phabricator.wikimedia.org/P78492 and previous config saved to /var/cache/conftool/dbconfig/20250619-223332-marostegui.json |
[production] |
| 22:18 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1227', diff saved to https://phabricator.wikimedia.org/P78491 and previous config saved to /var/cache/conftool/dbconfig/20250619-221824-marostegui.json |
[production] |
| 22:03 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1227 (T396130)', diff saved to https://phabricator.wikimedia.org/P78490 and previous config saved to /var/cache/conftool/dbconfig/20250619-220317-marostegui.json |
[production] |
| 21:45 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1227 (T396130)', diff saved to https://phabricator.wikimedia.org/P78489 and previous config saved to /var/cache/conftool/dbconfig/20250619-214540-marostegui.json |
[production] |
| 21:45 |
<marostegui@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1227.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:45 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1202 (T396130)', diff saved to https://phabricator.wikimedia.org/P78488 and previous config saved to /var/cache/conftool/dbconfig/20250619-214517-marostegui.json |
[production] |
| 21:39 |
<kostajh> |
mwscript-k8s -f --comment="T397483" -- updateCollation.php --wiki=plwikiquote --previous-collation=uppercase |
[production] |
| 21:30 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1202', diff saved to https://phabricator.wikimedia.org/P78487 and previous config saved to /var/cache/conftool/dbconfig/20250619-213010-marostegui.json |
[production] |