|
2021-03-11
ยง
|
| 14:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 60%: Repool db1113:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14796 and previous config saved to /var/cache/conftool/dbconfig/20210311-145204-root.json |
[production] |
| 14:50 |
<klausman> |
restarting pybal on lvs1016 T272918 |
[production] |
| 14:49 |
<klausman> |
restarting pybal on lvs2010 T272918 |
[production] |
| 14:46 |
<moritzm> |
installing openssl (1.1) security updates for stretch |
[production] |
| 14:37 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 30%: Repool db1113:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14795 and previous config saved to /var/cache/conftool/dbconfig/20210311-143700-root.json |
[production] |
| 14:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 10%: Repool db1113:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14794 and previous config saved to /var/cache/conftool/dbconfig/20210311-142157-root.json |
[production] |
| 14:07 |
<godog> |
swift eqiad-prod: decrease weight for SSDs on ms-be[1019-1026] - T272836 |
[production] |
| 14:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1113:3315 for schema change', diff saved to https://phabricator.wikimedia.org/P14793 and previous config saved to /var/cache/conftool/dbconfig/20210311-140526-marostegui.json |
[production] |
| 14:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1130 (re)pooling @ 100%: Repool db1130 after schema change', diff saved to https://phabricator.wikimedia.org/P14792 and previous config saved to /var/cache/conftool/dbconfig/20210311-140328-root.json |
[production] |
| 14:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Pool db2149 into s3', diff saved to https://phabricator.wikimedia.org/P14791 and previous config saved to /var/cache/conftool/dbconfig/20210311-140119-marostegui.json |
[production] |
| 13:56 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1063.eqiad.wmnet |
[production] |
| 13:51 |
<Majavah> |
shut down deployment-db06, now unused T277070 |
[releng] |
| 13:48 |
<Majavah> |
set deployment-db07 as r/w T277070 |
[releng] |
| 13:48 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1063.eqiad.wmnet |
[production] |
| 13:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1130 (re)pooling @ 60%: Repool db1130 after schema change', diff saved to https://phabricator.wikimedia.org/P14790 and previous config saved to /var/cache/conftool/dbconfig/20210311-134825-root.json |
[production] |
| 13:48 |
<Majavah> |
stop mariadb to ensure reads have stopped on deployment-db06 T277070 |
[releng] |
| 13:47 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1062.eqiad.wmnet |
[production] |
| 13:41 |
<Majavah> |
stop slave on deployment-db06 T276968 |
[releng] |
| 13:39 |
<jayme@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'production' . |
[production] |
| 13:39 |
<jayme@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . |
[production] |
| 13:39 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1062.eqiad.wmnet |
[production] |
| 13:37 |
<Majavah> |
make deployment-db06 and deployment-db08 be replicas of deployment-db07 T277070 |
[releng] |
| 13:36 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1061.eqiad.wmnet |
[production] |
| 13:34 |
<Majavah> |
stop and reset slave on deployment-db07 T277070 |
[releng] |
| 13:33 |
<jayme@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . |
[production] |
| 13:33 |
<jayme@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'production' . |
[production] |
| 13:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1130 (re)pooling @ 30%: Repool db1130 after schema change', diff saved to https://phabricator.wikimedia.org/P14789 and previous config saved to /var/cache/conftool/dbconfig/20210311-133321-root.json |
[production] |
| 13:32 |
<Majavah> |
set deployment-db06 as read only T277070 |
[releng] |
| 13:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1130 (re)pooling @ 10%: Repool db1130 after schema change', diff saved to https://phabricator.wikimedia.org/P14788 and previous config saved to /var/cache/conftool/dbconfig/20210311-131818-root.json |
[production] |
| 13:11 |
<arturo> |
add misctools 1.37 to buster-tools|toolsbeta aptly repo for T275865 |
[tools] |
| 13:10 |
<arturo> |
add jobutils 1.40 to buster-tools aptly repo for T275865 |
[tools] |
| 13:04 |
<moritzm> |
installing openssl1.0 security updates on stretch |
[production] |
| 13:03 |
<arturo> |
copy python-mwclient 0.8.4-1 from stretch-wikimedia to buster-wikimedia for T275865 |
[production] |
| 13:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1130 for schema change', diff saved to https://phabricator.wikimedia.org/P14787 and previous config saved to /var/cache/conftool/dbconfig/20210311-130208-marostegui.json |
[production] |
| 13:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1144:3315 (re)pooling @ 100%: Repool db1144:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14786 and previous config saved to /var/cache/conftool/dbconfig/20210311-130103-root.json |
[production] |
| 13:00 |
<hnowlan> |
imported cassandra_2.2.6-wmf5 to buster-wikimedia |
[production] |
| 12:49 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1061.eqiad.wmnet |
[production] |
| 12:46 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1060.eqiad.wmnet |
[production] |
| 12:46 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1144:3315 (re)pooling @ 60%: Repool db1144:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14785 and previous config saved to /var/cache/conftool/dbconfig/20210311-124559-root.json |
[production] |
| 12:39 |
<hnowlan> |
imported cassandra_2.2.6-wmf1 to buster-wikimedia |
[production] |
| 12:37 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1060.eqiad.wmnet |
[production] |
| 12:33 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1059.eqiad.wmnet |
[production] |
| 12:33 |
<arturo> |
livehacking puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/667144 for T275865 |
[toolsbeta] |
| 12:30 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1144:3315 (re)pooling @ 30%: Repool db1144:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14783 and previous config saved to /var/cache/conftool/dbconfig/20210311-123056-root.json |
[production] |
| 12:30 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1059.eqiad.wmnet |
[production] |
| 12:27 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1058.eqiad.wmnet |
[production] |
| 12:22 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1058.eqiad.wmnet |
[production] |
| 12:22 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1057.eqiad.wmnet |
[production] |
| 12:17 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1057.eqiad.wmnet |
[production] |
| 12:17 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1056.eqiad.wmnet |
[production] |