|
2021-03-11
ยง
|
| 18:56 |
<hnowlan@deploy1002> |
Started deploy [restbase/deploy@6f0fe23]: Remove internal ratelimits that were causing service proxy issues |
[production] |
| 18:47 |
<tgr_> |
running mwscript extensions/GrowthExperiments/maintenance/importOresTopics.php testwiki --count 1000 --verbose --wikiId enwiki --apiUrl 'https://en.wikipedia.org/w/api.php' |
[production] |
| 17:31 |
<effie> |
install mecached 1.6.6-1 on mwdebug1001 |
[production] |
| 16:26 |
<effie> |
upgrade memcached on mc1021, mc2021 |
[production] |
| 16:20 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 16:14 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 16:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1110 (re)pooling @ 100%: Repool db1110 after schema change', diff saved to https://phabricator.wikimedia.org/P14802 and previous config saved to /var/cache/conftool/dbconfig/20210311-161138-root.json |
[production] |
| 15:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1110 (re)pooling @ 60%: Repool db1110 after schema change', diff saved to https://phabricator.wikimedia.org/P14801 and previous config saved to /var/cache/conftool/dbconfig/20210311-155635-root.json |
[production] |
| 15:53 |
<cmjohnson1> |
updating firmware wdqs1009 T274751 |
[production] |
| 15:41 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1110 (re)pooling @ 30%: Repool db1110 after schema change', diff saved to https://phabricator.wikimedia.org/P14800 and previous config saved to /var/cache/conftool/dbconfig/20210311-154131-root.json |
[production] |
| 15:35 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 15:30 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 15:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1110 (re)pooling @ 10%: Repool db1110 after schema change', diff saved to https://phabricator.wikimedia.org/P14799 and previous config saved to /var/cache/conftool/dbconfig/20210311-152627-root.json |
[production] |
| 15:14 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1110 for schema change', diff saved to https://phabricator.wikimedia.org/P14798 and previous config saved to /var/cache/conftool/dbconfig/20210311-151435-marostegui.json |
[production] |
| 15:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 100%: Repool db1113:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14797 and previous config saved to /var/cache/conftool/dbconfig/20210311-150707-root.json |
[production] |
| 15:02 |
<klausman> |
restarting pybal on lvs1015 T272918 |
[production] |
| 14:55 |
<klausman> |
restarting pybal on lvs2009 T272918 |
[production] |
| 14:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 60%: Repool db1113:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14796 and previous config saved to /var/cache/conftool/dbconfig/20210311-145204-root.json |
[production] |
| 14:50 |
<klausman> |
restarting pybal on lvs1016 T272918 |
[production] |
| 14:49 |
<klausman> |
restarting pybal on lvs2010 T272918 |
[production] |
| 14:46 |
<moritzm> |
installing openssl (1.1) security updates for stretch |
[production] |
| 14:37 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 30%: Repool db1113:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14795 and previous config saved to /var/cache/conftool/dbconfig/20210311-143700-root.json |
[production] |
| 14:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 10%: Repool db1113:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14794 and previous config saved to /var/cache/conftool/dbconfig/20210311-142157-root.json |
[production] |
| 14:07 |
<godog> |
swift eqiad-prod: decrease weight for SSDs on ms-be[1019-1026] - T272836 |
[production] |
| 14:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1113:3315 for schema change', diff saved to https://phabricator.wikimedia.org/P14793 and previous config saved to /var/cache/conftool/dbconfig/20210311-140526-marostegui.json |
[production] |
| 14:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1130 (re)pooling @ 100%: Repool db1130 after schema change', diff saved to https://phabricator.wikimedia.org/P14792 and previous config saved to /var/cache/conftool/dbconfig/20210311-140328-root.json |
[production] |
| 14:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Pool db2149 into s3', diff saved to https://phabricator.wikimedia.org/P14791 and previous config saved to /var/cache/conftool/dbconfig/20210311-140119-marostegui.json |
[production] |
| 13:56 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1063.eqiad.wmnet |
[production] |
| 13:48 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1063.eqiad.wmnet |
[production] |
| 13:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1130 (re)pooling @ 60%: Repool db1130 after schema change', diff saved to https://phabricator.wikimedia.org/P14790 and previous config saved to /var/cache/conftool/dbconfig/20210311-134825-root.json |
[production] |
| 13:47 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1062.eqiad.wmnet |
[production] |
| 13:39 |
<jayme@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'production' . |
[production] |
| 13:39 |
<jayme@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . |
[production] |
| 13:39 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1062.eqiad.wmnet |
[production] |
| 13:36 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1061.eqiad.wmnet |
[production] |
| 13:33 |
<jayme@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . |
[production] |
| 13:33 |
<jayme@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'production' . |
[production] |
| 13:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1130 (re)pooling @ 30%: Repool db1130 after schema change', diff saved to https://phabricator.wikimedia.org/P14789 and previous config saved to /var/cache/conftool/dbconfig/20210311-133321-root.json |
[production] |
| 13:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1130 (re)pooling @ 10%: Repool db1130 after schema change', diff saved to https://phabricator.wikimedia.org/P14788 and previous config saved to /var/cache/conftool/dbconfig/20210311-131818-root.json |
[production] |
| 13:04 |
<moritzm> |
installing openssl1.0 security updates on stretch |
[production] |
| 13:03 |
<arturo> |
copy python-mwclient 0.8.4-1 from stretch-wikimedia to buster-wikimedia for T275865 |
[production] |
| 13:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1130 for schema change', diff saved to https://phabricator.wikimedia.org/P14787 and previous config saved to /var/cache/conftool/dbconfig/20210311-130208-marostegui.json |
[production] |
| 13:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1144:3315 (re)pooling @ 100%: Repool db1144:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14786 and previous config saved to /var/cache/conftool/dbconfig/20210311-130103-root.json |
[production] |
| 13:00 |
<hnowlan> |
imported cassandra_2.2.6-wmf5 to buster-wikimedia |
[production] |
| 12:49 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1061.eqiad.wmnet |
[production] |
| 12:46 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1060.eqiad.wmnet |
[production] |
| 12:46 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1144:3315 (re)pooling @ 60%: Repool db1144:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14785 and previous config saved to /var/cache/conftool/dbconfig/20210311-124559-root.json |
[production] |
| 12:39 |
<hnowlan> |
imported cassandra_2.2.6-wmf1 to buster-wikimedia |
[production] |
| 12:37 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1060.eqiad.wmnet |
[production] |
| 12:33 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1059.eqiad.wmnet |
[production] |