|
2022-03-14
ยง
|
| 13:43 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1179 (T298563)', diff saved to https://phabricator.wikimedia.org/P22423 and previous config saved to /var/cache/conftool/dbconfig/20220314-134356-marostegui.json |
[production] |
| 13:43 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 13:43 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudvirt1016.eqiad.wmnet with OS bullseye |
[production] |
| 13:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1147', diff saved to https://phabricator.wikimedia.org/P22422 and previous config saved to /var/cache/conftool/dbconfig/20220314-134231-marostegui.json |
[production] |
| 13:39 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 13:39 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 13:36 |
<Emperor> |
restarting swift-proxy on ms-fe100[5-8] to update config to know about new eqiad frontends T303698 |
[production] |
| 13:34 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 13:28 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P22421 and previous config saved to /var/cache/conftool/dbconfig/20220314-132849-marostegui.json |
[production] |
| 13:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1147', diff saved to https://phabricator.wikimedia.org/P22420 and previous config saved to /var/cache/conftool/dbconfig/20220314-132726-marostegui.json |
[production] |
| 13:25 |
<dcausse> |
restarting blazegraph on wdqs1006 (jvm stuck for 10hours) |
[production] |
| 13:24 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 13:23 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 13:23 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 13:22 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 13:20 |
<urbanecm@deploy1002> |
Synchronized static/images/project-logos/: 3fa9683: Delete huwiki 500k milestone logo files (T301923) (duration: 00m 49s) |
[production] |
| 13:18 |
<urbanecm@deploy1002> |
Synchronized wmf-config/logos.php: 3c2c8b0cca4e48f572abd3812594097a33e64379: Stop using huwiki 500k milestone logo (T301923) (duration: 00m 48s) |
[production] |
| 13:17 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 13:13 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P22419 and previous config saved to /var/cache/conftool/dbconfig/20220314-131344-marostegui.json |
[production] |
| 13:13 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 13:13 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 13:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1147 (T298294)', diff saved to https://phabricator.wikimedia.org/P22418 and previous config saved to /var/cache/conftool/dbconfig/20220314-131220-marostegui.json |
[production] |
| 13:11 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 13:10 |
<urbanecm@deploy1002> |
Synchronized wmf-config/wikitech.php: 95f376a: wikitech: migrate wmf* to wmg* (T45956) (duration: 00m 48s) |
[production] |
| 12:58 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1179 (T298563)', diff saved to https://phabricator.wikimedia.org/P22417 and previous config saved to /var/cache/conftool/dbconfig/20220314-125839-marostegui.json |
[production] |
| 12:49 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1147 (T298294)', diff saved to https://phabricator.wikimedia.org/P22416 and previous config saved to /var/cache/conftool/dbconfig/20220314-124911-marostegui.json |
[production] |
| 12:49 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1147.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:49 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1147.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:49 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1144:3314 (T298294)', diff saved to https://phabricator.wikimedia.org/P22415 and previous config saved to /var/cache/conftool/dbconfig/20220314-124902-marostegui.json |
[production] |
| 12:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1144:3314', diff saved to https://phabricator.wikimedia.org/P22414 and previous config saved to /var/cache/conftool/dbconfig/20220314-123357-marostegui.json |
[production] |
| 12:19 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1179 (T298563)', diff saved to https://phabricator.wikimedia.org/P22413 and previous config saved to /var/cache/conftool/dbconfig/20220314-121937-marostegui.json |
[production] |
| 12:19 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1179.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:19 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1179.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1144:3314', diff saved to https://phabricator.wikimedia.org/P22412 and previous config saved to /var/cache/conftool/dbconfig/20220314-121852-marostegui.json |
[production] |
| 12:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1144:3314 (T298294)', diff saved to https://phabricator.wikimedia.org/P22411 and previous config saved to /var/cache/conftool/dbconfig/20220314-120347-marostegui.json |
[production] |
| 11:55 |
<moritzm> |
restarting nginx on archiva1002 to pick up security updates |
[production] |
| 11:53 |
<moritzm> |
restarting apache2 on matomo1002 to pick up security updates |
[production] |
| 11:43 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1144:3314 (T298294)', diff saved to https://phabricator.wikimedia.org/P22410 and previous config saved to /var/cache/conftool/dbconfig/20220314-114312-marostegui.json |
[production] |
| 11:43 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1144.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:43 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1144.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:43 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1143 (T298294)', diff saved to https://phabricator.wikimedia.org/P22409 and previous config saved to /var/cache/conftool/dbconfig/20220314-114305-marostegui.json |
[production] |
| 11:28 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1143', diff saved to https://phabricator.wikimedia.org/P22408 and previous config saved to /var/cache/conftool/dbconfig/20220314-112759-marostegui.json |
[production] |
| 11:21 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:21 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1175 (T298563)', diff saved to https://phabricator.wikimedia.org/P22407 and previous config saved to /var/cache/conftool/dbconfig/20220314-112117-marostegui.json |
[production] |
| 11:18 |
<mbsantos@deploy1002> |
Finished deploy [kartotherian/deploy@c8a9efd] (eqiad): Enable mirroring on eqiad with 50% of the traffic (duration: 02m 38s) |
[production] |
| 11:15 |
<mbsantos@deploy1002> |
Started deploy [kartotherian/deploy@c8a9efd] (eqiad): Enable mirroring on eqiad with 50% of the traffic |
[production] |
| 11:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1143', diff saved to https://phabricator.wikimedia.org/P22406 and previous config saved to /var/cache/conftool/dbconfig/20220314-111255-marostegui.json |
[production] |
| 11:12 |
<mbsantos@deploy1002> |
Finished deploy [kartotherian/deploy@594f1d5] (eqiad): Revert "Revert "Mirror 100% of request to tegola in eqiad"" (duration: 07m 01s) |
[production] |
| 11:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P22405 and previous config saved to /var/cache/conftool/dbconfig/20220314-110612-marostegui.json |
[production] |