|
2024-10-21
ยง
|
| 14:32 |
<herron@deploy2002> |
helmfile [aux-k8s-eqiad] START helmfile.d/aus-k8s-eqiad-services/jaeger: apply |
[production] |
| 14:32 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db2192 (T376905)', diff saved to https://phabricator.wikimedia.org/P70382 and previous config saved to /var/cache/conftool/dbconfig/20241021-143108-ladsgroup.json |
[production] |
| 14:31 |
<ladsgroup@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2192.codfw.wmnet with reason: Maintenance |
[production] |
| 14:31 |
<ladsgroup@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2192.codfw.wmnet with reason: Maintenance |
[production] |
| 14:30 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2178 (T376905)', diff saved to https://phabricator.wikimedia.org/P70381 and previous config saved to /var/cache/conftool/dbconfig/20241021-143042-ladsgroup.json |
[production] |
| 14:29 |
<moritzm> |
installing PHP 8.2 security updates |
[production] |
| 14:15 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2178', diff saved to https://phabricator.wikimedia.org/P70380 and previous config saved to /var/cache/conftool/dbconfig/20241021-141535-ladsgroup.json |
[production] |
| 14:15 |
<Lucas_WMDE> |
UTC afternoon backport+config window done |
[production] |
| 14:10 |
<stran@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1081415|Disable IP reveal rights for local metawiki groups (T377584)]], [[gerrit:1081138|Set redirect wiki for Special:GlobalContributions (T376612)]], [[gerrit:1080227|temp accounts: Make temp accounts known on metawiki (T376132)]] (duration: 14m 55s) |
[production] |
| 14:05 |
<stran@deploy2002> |
stran, kharlan: Continuing with sync |
[production] |
| 14:00 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2178', diff saved to https://phabricator.wikimedia.org/P70379 and previous config saved to /var/cache/conftool/dbconfig/20241021-140028-ladsgroup.json |
[production] |
| 13:57 |
<stran@deploy2002> |
stran, kharlan: Backport for [[gerrit:1081415|Disable IP reveal rights for local metawiki groups (T377584)]], [[gerrit:1081138|Set redirect wiki for Special:GlobalContributions (T376612)]], [[gerrit:1080227|temp accounts: Make temp accounts known on metawiki (T376132)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 13:57 |
<elukey@cumin1002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti2035.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART |
[production] |
| 13:55 |
<stran@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1081415|Disable IP reveal rights for local metawiki groups (T377584)]], [[gerrit:1081138|Set redirect wiki for Special:GlobalContributions (T376612)]], [[gerrit:1080227|temp accounts: Make temp accounts known on metawiki (T376132)]] |
[production] |
| 13:54 |
<elukey@cumin1002> |
START - Cookbook sre.hosts.provision for host ganeti2035.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART |
[production] |
| 13:53 |
<elukey@cumin1002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti2035.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART |
[production] |
| 13:50 |
<elukey@cumin1002> |
START - Cookbook sre.hosts.provision for host ganeti2035.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART |
[production] |
| 13:50 |
<stran@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1080250|Apply wmf-specific protected vars rights access (T369610)]] (duration: 08m 53s) |
[production] |
| 13:45 |
<stran@deploy2002> |
stran: Continuing with sync |
[production] |
| 13:45 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2178 (T376905)', diff saved to https://phabricator.wikimedia.org/P70378 and previous config saved to /var/cache/conftool/dbconfig/20241021-134521-ladsgroup.json |
[production] |
| 13:43 |
<stran@deploy2002> |
stran: Backport for [[gerrit:1080250|Apply wmf-specific protected vars rights access (T369610)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 13:41 |
<stran@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1080250|Apply wmf-specific protected vars rights access (T369610)]] |
[production] |
| 13:37 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db2178 (T376905)', diff saved to https://phabricator.wikimedia.org/P70377 and previous config saved to /var/cache/conftool/dbconfig/20241021-133619-ladsgroup.json |
[production] |
| 13:37 |
<ladsgroup@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2178.codfw.wmnet with reason: Maintenance |
[production] |
| 13:36 |
<ladsgroup@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2178.codfw.wmnet with reason: Maintenance |
[production] |
| 13:36 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2171 (T376905)', diff saved to https://phabricator.wikimedia.org/P70376 and previous config saved to /var/cache/conftool/dbconfig/20241021-133552-ladsgroup.json |
[production] |
| 13:35 |
<jelto@deploy2002> |
helmfile [codfw] DONE helmfile.d/services/miscweb: apply |
[production] |
| 13:34 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7002.magru.wmnet |
[production] |
| 13:34 |
<lucaswerkmeister-wmde@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1081973|Revert "Enable CampaignEvents collaboration list in testwiki and test2wiki"]] (duration: 08m 20s) |
[production] |
| 13:33 |
<jelto@deploy2002> |
helmfile [codfw] START helmfile.d/services/miscweb: apply |
[production] |
| 13:33 |
<inflatador> |
bking@stat1009,stat1010.mgmt racadm>>racadm set BIOS.MemSettings.NodeInterleave Enabled && racadm jobqueue create BIOS.Setup.1-1 T376813 |
[production] |
| 13:32 |
<jelto@deploy2002> |
helmfile [eqiad] DONE helmfile.d/services/miscweb: apply |
[production] |
| 13:30 |
<arnaudb@cumin1002> |
START - Cookbook sre.mysql.clone of db2172.codfw.wmnet onto db2240.codfw.wmnet |
[production] |
| 13:29 |
<jelto@deploy2002> |
helmfile [eqiad] START helmfile.d/services/miscweb: apply |
[production] |
| 13:29 |
<lucaswerkmeister-wmde@deploy2002> |
lucaswerkmeister-wmde, trainbranchbot: Continuing with sync |
[production] |
| 13:28 |
<jelto@deploy2002> |
helmfile [staging] DONE helmfile.d/services/miscweb: apply |
[production] |
| 13:28 |
<lucaswerkmeister-wmde@deploy2002> |
lucaswerkmeister-wmde, trainbranchbot: Backport for [[gerrit:1081973|Revert "Enable CampaignEvents collaboration list in testwiki and test2wiki"]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 13:27 |
<jelto@deploy2002> |
helmfile [staging] START helmfile.d/services/miscweb: apply |
[production] |
| 13:26 |
<lucaswerkmeister-wmde@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1081973|Revert "Enable CampaignEvents collaboration list in testwiki and test2wiki"]] |
[production] |
| 13:25 |
<inflatador> |
bking@stat1008.mgmt racadm>>racadm jobqueue create BIOS.Setup.1-1 |
[production] |
| 13:24 |
<inflatador> |
bking@stat1008.mgmt racadm>>racadm set BIOS.MemSettings.NodeInterleave Enabled T376813 |
[production] |
| 13:24 |
<lucaswerkmeister-wmde@deploy2002> |
Sync cancelled. |
[production] |
| 13:23 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Cloning db2172 in db2240 for T373579', diff saved to https://phabricator.wikimedia.org/P70375 and previous config saved to /var/cache/conftool/dbconfig/20241021-132351-arnaudb.json |
[production] |
| 13:22 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2240.codfw.wmnet with reason: provisionning db2240.codfw.wmnet - T373579 |
[production] |
| 13:22 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2240.codfw.wmnet with reason: provisionning db2240.codfw.wmnet - T373579 |
[production] |
| 13:22 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2172.codfw.wmnet with reason: provisionning db2240.codfw.wmnet - T373579 |
[production] |
| 13:22 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2172.codfw.wmnet with reason: provisionning db2240.codfw.wmnet - T373579 |
[production] |
| 13:20 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2171', diff saved to https://phabricator.wikimedia.org/P70374 and previous config saved to /var/cache/conftool/dbconfig/20241021-132045-ladsgroup.json |
[production] |
| 13:17 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Depool db2172 to clone on db2240 T373579', diff saved to https://phabricator.wikimedia.org/P70373 and previous config saved to /var/cache/conftool/dbconfig/20241021-131750-arnaudb.json |
[production] |
| 13:12 |
<jelto@cumin1002> |
END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1003.wikimedia.org with reason: test Ide32aae22b53884a76ab10ee4504052cbe777735 with dummy upgrade |
[production] |