|
2024-02-02
§
|
| 09:35 |
<akosiaris@deploy2002> |
helmfile [eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 09:27 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2137:3314 (T355609)', diff saved to https://phabricator.wikimedia.org/P56103 and previous config saved to /var/cache/conftool/dbconfig/20240202-092752-marostegui.json |
[production] |
| 09:18 |
<mvernon@cumin2002> |
END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies (exit_code=0) rolling restart_daemons on A:swift-fe |
[production] |
| 09:09 |
<mvernon@cumin2002> |
START - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies rolling restart_daemons on A:swift-fe |
[production] |
| 09:02 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db2137:3314 (T355609)', diff saved to https://phabricator.wikimedia.org/P56102 and previous config saved to /var/cache/conftool/dbconfig/20240202-090209-marostegui.json |
[production] |
| 09:02 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2137.codfw.wmnet with reason: Maintenance |
[production] |
| 09:01 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db2137.codfw.wmnet with reason: Maintenance |
[production] |
| 09:01 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2155 (T355609)', diff saved to https://phabricator.wikimedia.org/P56101 and previous config saved to /var/cache/conftool/dbconfig/20240202-090146-marostegui.json |
[production] |
| 08:57 |
<mvernon@cumin2002> |
END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies (exit_code=0) rolling restart_daemons on P{ms-fe1011.eqiad.wmnet} and (A:swift-fe or A:swift-fe-canary or A:swift-fe-codfw or A:swift-fe-eqiad) |
[production] |
| 08:57 |
<mvernon@cumin2002> |
START - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies rolling restart_daemons on P{ms-fe1011.eqiad.wmnet} and (A:swift-fe or A:swift-fe-canary or A:swift-fe-codfw or A:swift-fe-eqiad) |
[production] |
| 08:46 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2155', diff saved to https://phabricator.wikimedia.org/P56100 and previous config saved to /var/cache/conftool/dbconfig/20240202-084640-marostegui.json |
[production] |
| 08:31 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2155', diff saved to https://phabricator.wikimedia.org/P56099 and previous config saved to /var/cache/conftool/dbconfig/20240202-083133-marostegui.json |
[production] |
| 08:30 |
<tstarling@deploy2002> |
Synchronized wmf-config/CommonSettings.php: Enable UrlShortener QR code everywhere (T348487) (duration: 07m 23s) |
[production] |
| 08:16 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2155 (T355609)', diff saved to https://phabricator.wikimedia.org/P56098 and previous config saved to /var/cache/conftool/dbconfig/20240202-081626-marostegui.json |
[production] |
| 07:18 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast3007.wikimedia.org |
[production] |
| 07:15 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db2155 (T355609)', diff saved to https://phabricator.wikimedia.org/P56097 and previous config saved to /var/cache/conftool/dbconfig/20240202-071555-marostegui.json |
[production] |
| 07:15 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2187.codfw.wmnet with reason: Maintenance |
[production] |
| 07:15 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2187.codfw.wmnet with reason: Maintenance |
[production] |
| 07:15 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2155.codfw.wmnet with reason: Maintenance |
[production] |
| 07:15 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db2155.codfw.wmnet with reason: Maintenance |
[production] |
| 07:15 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2147 (T355609)', diff saved to https://phabricator.wikimedia.org/P56096 and previous config saved to /var/cache/conftool/dbconfig/20240202-071516-marostegui.json |
[production] |
| 07:12 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host bast3007.wikimedia.org |
[production] |
| 07:00 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2147', diff saved to https://phabricator.wikimedia.org/P56095 and previous config saved to /var/cache/conftool/dbconfig/20240202-070009-marostegui.json |
[production] |
| 06:58 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast6003.wikimedia.org |
[production] |
| 06:53 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host bast6003.wikimedia.org |
[production] |
| 06:45 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2147', diff saved to https://phabricator.wikimedia.org/P56094 and previous config saved to /var/cache/conftool/dbconfig/20240202-064502-marostegui.json |
[production] |
| 06:38 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Change db1163 weight', diff saved to https://phabricator.wikimedia.org/P56093 and previous config saved to /var/cache/conftool/dbconfig/20240202-063858-marostegui.json |
[production] |
| 06:38 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Change db1163 weight', diff saved to https://phabricator.wikimedia.org/P56092 and previous config saved to /var/cache/conftool/dbconfig/20240202-063844-marostegui.json |
[production] |
| 06:14 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db1106.eqiad.wmnet |
[production] |
| 06:14 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 06:14 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db1106.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1002" |
[production] |
| 06:13 |
<marostegui@cumin1002> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db1106.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1002" |
[production] |
| 06:11 |
<marostegui@cumin1002> |
START - Cookbook sre.dns.netbox |
[production] |
| 06:06 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.decommission for hosts db1106.eqiad.wmnet |
[production] |
| 06:05 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db2147 (T355609)', diff saved to https://phabricator.wikimedia.org/P56090 and previous config saved to /var/cache/conftool/dbconfig/20240202-060504-marostegui.json |
[production] |
| 06:04 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2147.codfw.wmnet with reason: Maintenance |
[production] |
| 06:04 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db2147.codfw.wmnet with reason: Maintenance |
[production] |
| 00:23 |
<thcipriani@deploy2002> |
Finished scap: Backport for [[gerrit:995120|wikitech: Update Gerrit blocking logic (T307558)]] (duration: 09m 06s) |
[production] |
| 00:16 |
<thcipriani@deploy2002> |
thcipriani and bd808: Continuing with sync |
[production] |
| 00:15 |
<thcipriani@deploy2002> |
thcipriani and bd808: Backport for [[gerrit:995120|wikitech: Update Gerrit blocking logic (T307558)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 00:14 |
<thcipriani@deploy2002> |
Started scap: Backport for [[gerrit:995120|wikitech: Update Gerrit blocking logic (T307558)]] |
[production] |
|
2024-02-01
§
|
| 22:54 |
<bking@cumin2002> |
conftool action : set/weight=10; selector: name=cloudelastic1010.eqiad.wmnet |
[production] |
| 22:52 |
<bking@cumin2002> |
conftool action : set/pooled=yes; selector: dc=eqiad,cluster=cloudelastic,name=cloudelastic1010.eqiad.wmnet |
[production] |
| 22:51 |
<bking@cumin2002> |
conftool action : set/pooled=yes; selector: dc=eqiad,cluster=cloudelastic,name=cloudelastic1003.wikimedia.org |
[production] |
| 22:51 |
<bking@cumin2002> |
conftool action : set/weight=10; selector: name=cloudelastic1010. |
[production] |
| 21:57 |
<mutante> |
LDAP - added wmdecyn to wmde and nda groups T355937 |
[production] |
| 21:38 |
<tchanders@deploy2002> |
Finished scap: Backport for [[gerrit:995059|Set $wgEnablePartialActionBlocks true for most wikis (T353495)]] (duration: 10m 04s) |
[production] |
| 21:32 |
<tchanders@deploy2002> |
tchanders: Continuing with sync |
[production] |
| 21:30 |
<tchanders@deploy2002> |
tchanders: Backport for [[gerrit:995059|Set $wgEnablePartialActionBlocks true for most wikis (T353495)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 21:28 |
<tchanders@deploy2002> |
Started scap: Backport for [[gerrit:995059|Set $wgEnablePartialActionBlocks true for most wikis (T353495)]] |
[production] |