|
2026-02-11
ยง
|
| 22:03 |
<cjming@deploy2002> |
cjming: Backport for [[gerrit:1238799|Fix contextual attributes when stream is set (T417091)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 22:01 |
<cjming@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1238799|Fix contextual attributes when stream is set (T417091)]] |
[production] |
| 21:53 |
<cjming@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1238814|Mock Experiment class in ExperimentTestKitchenManager unit test]] (duration: 06m 48s) |
[production] |
| 21:49 |
<cjming@deploy2002> |
cjming: Continuing with sync |
[production] |
| 21:48 |
<cjming@deploy2002> |
cjming: Backport for [[gerrit:1238814|Mock Experiment class in ExperimentTestKitchenManager unit test]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 21:46 |
<cjming@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1238814|Mock Experiment class in ExperimentTestKitchenManager unit test]] |
[production] |
| 21:34 |
<cjming@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1238072|Remove A/B test for hCaptcha editing (T410354)]] (duration: 06m 53s) |
[production] |
| 21:30 |
<cjming@deploy2002> |
kharlan, cjming: Continuing with sync |
[production] |
| 21:30 |
<cjming@deploy2002> |
kharlan, cjming: Backport for [[gerrit:1238072|Remove A/B test for hCaptcha editing (T410354)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 21:27 |
<cjming@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1238072|Remove A/B test for hCaptcha editing (T410354)]] |
[production] |
| 21:25 |
<cjming@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1234538|Configure rate limit class for global bots (T415588)]], [[gerrit:1235499|Remove the wgGlobalWatchlistWikibaseSite variable values (T415440)]] (duration: 07m 43s) |
[production] |
| 21:20 |
<cjming@deploy2002> |
cjming, matmarex, ikhitron: Continuing with sync |
[production] |
| 21:19 |
<cjming@deploy2002> |
cjming, matmarex, ikhitron: Backport for [[gerrit:1234538|Configure rate limit class for global bots (T415588)]], [[gerrit:1235499|Remove the wgGlobalWatchlistWikibaseSite variable values (T415440)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 21:17 |
<cjming@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1234538|Configure rate limit class for global bots (T415588)]], [[gerrit:1235499|Remove the wgGlobalWatchlistWikibaseSite variable values (T415440)]] |
[production] |
| 21:12 |
<ryankemper@cumin2002> |
END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) for hosts an-worker1132.eqiad.wmnet |
[production] |
| 21:12 |
<ryankemper@cumin2002> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 21:12 |
<ryankemper@cumin2002> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: an-worker1132.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - ryankemper@cumin2002" |
[production] |
| 21:12 |
<ryankemper@cumin2002> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: an-worker1132.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - ryankemper@cumin2002" |
[production] |
| 21:08 |
<kharlan@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1238709|IPoid: Add configurable connectTimeout (T416164)]], [[gerrit:1238710|IPoid: Add configurable connectTimeout (T416164)]] (duration: 08m 11s) |
[production] |
| 21:06 |
<ryankemper@cumin2002> |
START - Cookbook sre.dns.netbox |
[production] |
| 21:04 |
<kharlan@deploy2002> |
stran, kharlan: Continuing with sync |
[production] |
| 21:03 |
<kharlan@deploy2002> |
stran, kharlan: Backport for [[gerrit:1238709|IPoid: Add configurable connectTimeout (T416164)]], [[gerrit:1238710|IPoid: Add configurable connectTimeout (T416164)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 21:02 |
<ryankemper@cumin2002> |
START - Cookbook sre.hosts.decommission for hosts an-worker1132.eqiad.wmnet |
[production] |
| 21:00 |
<kharlan@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1238709|IPoid: Add configurable connectTimeout (T416164)]], [[gerrit:1238710|IPoid: Add configurable connectTimeout (T416164)]] |
[production] |
| 20:48 |
<ryankemper> |
T411919 Rebooting `an-worker1148` again. First reboot went great, but just double-checking, and also I'd removed a duplicate fstab entry so this will sanity check my change there |
[production] |
| 20:19 |
<ryankemper> |
T411919 Rebooting `an-worker1148` to see if it comes back up properly after having switched the RAID card |
[production] |
| 19:32 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Depooling db1178 (T410589)', diff saved to https://phabricator.wikimedia.org/P88784 and previous config saved to /var/cache/conftool/dbconfig/20260211-193220-ladsgroup.json |
[production] |
| 19:32 |
<ladsgroup@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:31 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1177 (T410589)', diff saved to https://phabricator.wikimedia.org/P88783 and previous config saved to /var/cache/conftool/dbconfig/20260211-193155-ladsgroup.json |
[production] |
| 19:21 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P88779 and previous config saved to /var/cache/conftool/dbconfig/20260211-192146-ladsgroup.json |
[production] |
| 19:11 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P88778 and previous config saved to /var/cache/conftool/dbconfig/20260211-191138-ladsgroup.json |
[production] |
| 19:01 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1177 (T410589)', diff saved to https://phabricator.wikimedia.org/P88777 and previous config saved to /var/cache/conftool/dbconfig/20260211-190130-ladsgroup.json |
[production] |
| 18:16 |
<fceratto@cumin1003> |
END (PASS) - Cookbook sre.mysql.update-replication (exit_code=0) |
[production] |
| 18:16 |
<fceratto@cumin1003> |
START - Cookbook sre.mysql.update-replication |
[production] |
| 18:15 |
<fceratto@cumin1003> |
END (PASS) - Cookbook sre.mysql.update-replication (exit_code=0) |
[production] |
| 18:15 |
<fceratto@cumin1003> |
START - Cookbook sre.mysql.update-replication |
[production] |
| 18:14 |
<fceratto@cumin1003> |
END (PASS) - Cookbook sre.mysql.update-replication (exit_code=0) |
[production] |
| 18:14 |
<fceratto@cumin1003> |
START - Cookbook sre.mysql.update-replication |
[production] |
| 18:14 |
<fceratto@cumin1003> |
END (PASS) - Cookbook sre.mysql.update-replication (exit_code=0) |
[production] |
| 18:14 |
<fceratto@cumin1003> |
START - Cookbook sre.mysql.update-replication |
[production] |
| 18:11 |
<fceratto@cumin1003> |
END (PASS) - Cookbook sre.mysql.update-replication (exit_code=0) |
[production] |
| 18:11 |
<fceratto@cumin1003> |
START - Cookbook sre.mysql.update-replication |
[production] |
| 18:10 |
<fceratto@cumin1003> |
END (PASS) - Cookbook sre.mysql.update-replication (exit_code=0) |
[production] |
| 18:10 |
<fceratto@cumin1003> |
START - Cookbook sre.mysql.update-replication |
[production] |
| 18:09 |
<fceratto@cumin1003> |
END (PASS) - Cookbook sre.mysql.update-replication (exit_code=0) |
[production] |
| 18:09 |
<fceratto@cumin1003> |
START - Cookbook sre.mysql.update-replication |
[production] |
| 18:08 |
<fceratto@cumin1003> |
END (PASS) - Cookbook sre.mysql.update-replication (exit_code=0) |
[production] |
| 18:08 |
<fceratto@cumin1003> |
START - Cookbook sre.mysql.update-replication |
[production] |
| 18:07 |
<fceratto@cumin1003> |
END (PASS) - Cookbook sre.mysql.update-replication (exit_code=0) |
[production] |
| 18:07 |
<fceratto@cumin1003> |
START - Cookbook sre.mysql.update-replication |
[production] |