|
2021-03-11
ยง
|
| 20:04 |
<brennen@deploy1002> |
rebuilt and synchronized wikiversions files: all wikis to 1.36.0-wmf.34 |
[production] |
| 19:59 |
<mutante> |
phab1001 - sudo systemctl start phabricator_clean_tmp_files (manually run after conversion from cron to timer, and it fails with permission issues) |
[production] |
| 19:55 |
<tgr_> |
T277173 running mwscript extensions/WikimediaMaintenance/createExtensionTables.php --wiki=testwiki GrowthExperiments |
[production] |
| 19:54 |
<tgr@deploy1002> |
Synchronized wmf-config/: Config: [[gerrit:670857|Configure GrowthExperiments Add Link settings, step 2 (T277173)]] (duration: 01m 08s) |
[production] |
| 19:43 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 19:30 |
<tgr@deploy1002> |
Synchronized wmf-config/: Config: [[gerrit:670887|Configure GrowthExperiments Add Link settings, step 1 (T277173)]] (duration: 01m 08s) |
[production] |
| 19:18 |
<tgr@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:668196|wikitech: enable BetaFeatures (T125941)]] (duration: 01m 08s) |
[production] |
| 19:13 |
<hnowlan@deploy1002> |
Finished deploy [restbase/deploy@6f0fe23]: Remove internal ratelimits that were causing service proxy issues (duration: 16m 25s) |
[production] |
| 18:56 |
<hnowlan@deploy1002> |
Started deploy [restbase/deploy@6f0fe23]: Remove internal ratelimits that were causing service proxy issues |
[production] |
| 18:52 |
<razzi> |
systemctl restart hadoop-hdfs-datanode on analytics1059 |
[analytics] |
| 18:50 |
<razzi> |
systemctl restart hadoop-yarn-nodemanager on analytics1059 |
[analytics] |
| 18:47 |
<tgr_> |
running mwscript extensions/GrowthExperiments/maintenance/importOresTopics.php testwiki --count 1000 --verbose --wikiId enwiki --apiUrl 'https://en.wikipedia.org/w/api.php' |
[production] |
| 18:38 |
<legoktm> |
upgrade pip in virtualenv and downgrade pymysql to 0.10 |
[tools.tutor] |
| 18:35 |
<razzi> |
apt-get install parted on analytics1059 |
[analytics] |
| 18:33 |
<bstorm> |
silenced alerts from deploymentprep for another 60 days |
[metricsinfra] |
| 17:40 |
<bstorm> |
deployed metrics-server:0.4.1 to kubernetes |
[tools] |
| 17:31 |
<effie> |
install mecached 1.6.6-1 on mwdebug1001 |
[production] |
| 16:57 |
<Majavah> |
copy a tarball of deployment-fluorine02 /home to deployment-mwlog01 root home dir, delete deployment-fluorine02 T276419 |
[releng] |
| 16:49 |
<Majavah> |
delete deployment-etcd-01 T276462 |
[releng] |
| 16:26 |
<effie> |
upgrade memcached on mc1021, mc2021 |
[production] |
| 16:21 |
<bstorm> |
add jobutils 1.40 and misctools 1.40 to stretch-tools |
[tools] |
| 16:20 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 16:14 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 16:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1110 (re)pooling @ 100%: Repool db1110 after schema change', diff saved to https://phabricator.wikimedia.org/P14802 and previous config saved to /var/cache/conftool/dbconfig/20210311-161138-root.json |
[production] |
| 15:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1110 (re)pooling @ 60%: Repool db1110 after schema change', diff saved to https://phabricator.wikimedia.org/P14801 and previous config saved to /var/cache/conftool/dbconfig/20210311-155635-root.json |
[production] |
| 15:53 |
<cmjohnson1> |
updating firmware wdqs1009 T274751 |
[production] |
| 15:41 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1110 (re)pooling @ 30%: Repool db1110 after schema change', diff saved to https://phabricator.wikimedia.org/P14800 and previous config saved to /var/cache/conftool/dbconfig/20210311-154131-root.json |
[production] |
| 15:35 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 15:34 |
<razzi> |
rebalance kafka partitions for webrequest_upload partition 17 |
[analytics] |
| 15:30 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 15:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1110 (re)pooling @ 10%: Repool db1110 after schema change', diff saved to https://phabricator.wikimedia.org/P14799 and previous config saved to /var/cache/conftool/dbconfig/20210311-152627-root.json |
[production] |
| 15:14 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1110 for schema change', diff saved to https://phabricator.wikimedia.org/P14798 and previous config saved to /var/cache/conftool/dbconfig/20210311-151435-marostegui.json |
[production] |
| 15:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 100%: Repool db1113:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14797 and previous config saved to /var/cache/conftool/dbconfig/20210311-150707-root.json |
[production] |
| 15:02 |
<klausman> |
restarting pybal on lvs1015 T272918 |
[production] |
| 14:55 |
<klausman> |
restarting pybal on lvs2009 T272918 |
[production] |
| 14:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 60%: Repool db1113:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14796 and previous config saved to /var/cache/conftool/dbconfig/20210311-145204-root.json |
[production] |
| 14:50 |
<klausman> |
restarting pybal on lvs1016 T272918 |
[production] |
| 14:49 |
<klausman> |
restarting pybal on lvs2010 T272918 |
[production] |
| 14:46 |
<moritzm> |
installing openssl (1.1) security updates for stretch |
[production] |
| 14:37 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 30%: Repool db1113:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14795 and previous config saved to /var/cache/conftool/dbconfig/20210311-143700-root.json |
[production] |
| 14:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 10%: Repool db1113:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14794 and previous config saved to /var/cache/conftool/dbconfig/20210311-142157-root.json |
[production] |
| 14:07 |
<godog> |
swift eqiad-prod: decrease weight for SSDs on ms-be[1019-1026] - T272836 |
[production] |
| 14:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1113:3315 for schema change', diff saved to https://phabricator.wikimedia.org/P14793 and previous config saved to /var/cache/conftool/dbconfig/20210311-140526-marostegui.json |
[production] |
| 14:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1130 (re)pooling @ 100%: Repool db1130 after schema change', diff saved to https://phabricator.wikimedia.org/P14792 and previous config saved to /var/cache/conftool/dbconfig/20210311-140328-root.json |
[production] |
| 14:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Pool db2149 into s3', diff saved to https://phabricator.wikimedia.org/P14791 and previous config saved to /var/cache/conftool/dbconfig/20210311-140119-marostegui.json |
[production] |
| 13:56 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1063.eqiad.wmnet |
[production] |
| 13:51 |
<Majavah> |
shut down deployment-db06, now unused T277070 |
[releng] |
| 13:48 |
<Majavah> |
set deployment-db07 as r/w T277070 |
[releng] |
| 13:48 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1063.eqiad.wmnet |
[production] |
| 13:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1130 (re)pooling @ 60%: Repool db1130 after schema change', diff saved to https://phabricator.wikimedia.org/P14790 and previous config saved to /var/cache/conftool/dbconfig/20210311-134825-root.json |
[production] |