|
2020-01-03
§
|
| 18:08 |
<bd808> |
Migrating to new kubernetes cluster |
[tools.bridgebot] |
| 17:46 |
<bstorm_> |
stashed uncommitted changes on the puppetmaster because they seem to be things that are already merged |
[toolsbeta] |
| 17:44 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Repool db2084 instances T241103', diff saved to https://phabricator.wikimedia.org/P10035 and previous config saved to /var/cache/conftool/dbconfig/20200103-174447-jynus.json |
[production] |
| 16:54 |
<ejegg> |
updated fundraising CiviCRM from 217a1f8c63 to 5642a92223 |
[production] |
| 16:48 |
<bstorm_> |
updated the ValidatingWebhookConfiguration for the ingress admission controller to the working settings |
[tools] |
| 16:36 |
<jynus> |
stopping db2084 |
[production] |
| 15:04 |
<marostegui> |
Upgrade db2107 |
[production] |
| 14:58 |
<marostegui> |
Deploy schema changes on s2 and s4 eqiad hosts T234052 |
[production] |
| 14:56 |
<jbond42> |
clean up old /etc/apt/preferences.d/smartmontools.pref file |
[production] |
| 14:48 |
<jbond42> |
clean up old /etc/apt/preferences.d/puppet_all.pref file |
[production] |
| 14:45 |
<jbond42> |
clean up old /etc/apt/preferences.d/facter.pref file |
[production] |
| 14:15 |
<Urbanecm> |
Run undelete.php on a couple of pages at plwikisource per T241824 |
[production] |
| 13:50 |
<marostegui> |
Deploy schema change on s4 codfw (lag will appear on codfw s4) - T234052 |
[production] |
| 13:46 |
<moritzm> |
restarting exim on MXes to pick up SASL security update |
[production] |
| 11:51 |
<arturo> |
[new k8s] deploy cadvisor as in https://gerrit.wikimedia.org/r/c/operations/puppet/+/561654 (T237643) |
[tools] |
| 11:27 |
<arturo> |
[new k8s] cadvisor is running in the metrics namespace now (T237643) |
[toolsbeta] |
| 11:21 |
<arturo> |
upload k8s.gcr.io/cadvisor:v0.30.2 docker image to the docker registry as docker-registry.tools.wmflabs.org/cadvisor:0.30.2 for T237643 |
[tools] |
| 11:00 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1074 after schema change', diff saved to https://phabricator.wikimedia.org/P10033 and previous config saved to /var/cache/conftool/dbconfig/20200103-110028-marostegui.json |
[production] |
| 10:30 |
<arturo> |
increase CPU quota to 18 cores T241740 * |
[etytree] |
| 10:29 |
<arturo> |
increase CPU quota to 18 cores T241716 |
[etytree] |
| 10:20 |
<moritzm> |
restarting apache on cloudmetrics* to pick up SASL security update |
[production] |
| 10:11 |
<moritzm> |
installing cyrus-sasl2 security updates on stretch/buster |
[production] |
| 09:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1074 schema change', diff saved to https://phabricator.wikimedia.org/P10032 and previous config saved to /var/cache/conftool/dbconfig/20200103-094252-marostegui.json |
[production] |
| 09:38 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1076 after schema change', diff saved to https://phabricator.wikimedia.org/P10031 and previous config saved to /var/cache/conftool/dbconfig/20200103-093829-marostegui.json |
[production] |
| 09:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1076 schema change', diff saved to https://phabricator.wikimedia.org/P10030 and previous config saved to /var/cache/conftool/dbconfig/20200103-092107-marostegui.json |
[production] |
| 08:17 |
<marostegui> |
Deploy schema change on labswiki (wikitech) T234052 |
[production] |
| 07:51 |
<legoktm> |
switched over to python3.7 and new k8s cluster (`kubectl config use-context toolforge`) |
[tools.apt-browser] |
| 07:36 |
<legoktm> |
switched to python3.7 webservice |
[tools.ldap] |
| 07:24 |
<legoktm> |
switched over to new k8s cluster (`kubectl config use-context toolforge`) |
[tools.phpinfo] |
| 07:09 |
<marostegui> |
Deploy schema change on s2 codfw master, lag will appear on codfw - T234052 |
[production] |
| 06:57 |
<marostegui> |
Deploy schema change on s6 eqiad hosts - T234052 |
[production] |
| 06:23 |
<marostegui> |
Deploy schema change on db2089:3316 |
[production] |
| 06:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db2089:3316 - T239453', diff saved to https://phabricator.wikimedia.org/P10029 and previous config saved to /var/cache/conftool/dbconfig/20200103-062242-marostegui.json |
[production] |
| 06:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db2087:3316 - T239453', diff saved to https://phabricator.wikimedia.org/P10028 and previous config saved to /var/cache/conftool/dbconfig/20200103-062148-marostegui.json |
[production] |
| 03:35 |
<legoktm> |
switched over to new k8s cluster (`kubectl config use-context toolforge`) |
[tools.ldap] |
| 03:04 |
<bd808> |
Really rebuilding all {jessie,stretch,buster}-sssd images. Last time I forgot to actually update the git clone. |
[tools] |
| 00:47 |
<paladox> |
puppet cert generate puppetmaster-1001.devtools.eqiad.wmflabs |
[devtools] |
| 00:30 |
<paladox> |
set puppetmaster: puppetmaster-1001.devtools.eqiad.wmflabs in hiera |
[devtools] |
| 00:11 |
<bd808> |
Rebuiliding all stretch-ssd Docker images to pick up busybox |
[tools] |
|
2020-01-02
§
|
| 23:54 |
<bd808> |
Rebuiliding all buster-ssd Docker images to pick up busybox |
[tools] |
| 23:42 |
<paladox> |
puppetmaster-1001: ln -s /var/lib/puppet/ssl/private_keys/puppetmaster-1001.devtools.eqiad.wmflabs.pem /var/lib/puppet/server/ssl/private_keys/puppetmaster-1001.devtools.eqiad.wmflabs.pem |
[devtools] |
| 23:40 |
<paladox> |
puppetmaster-1001: ln -s /var/lib/puppet/ssl/certs/puppetmaster-1001.devtools.eqiad.wmflabs.pem /var/lib/puppet/server/ssl/certs/puppetmaster-1001.devtools.eqiad.wmflabs.pem |
[devtools] |
| 23:35 |
<mutante> |
attempting to create local puppetmaster (formerly puppetmaster::self) |
[devtools] |
| 23:34 |
<mutante> |
cloud-puppetmaster-01 puppet cert clean puppetmaster-1001.devtools.eqiad.wmflabs |
[cloudinfra] |
| 23:33 |
<ejegg> |
updated Fundraising CiviCRM from d534f4e966 to 217a1f8c63 |
[production] |
| 23:09 |
<ejegg> |
updated Fundraising CiviCRM from 6936aa0262 to d534f4e966 |
[production] |
| 22:44 |
<ejegg> |
updated fundraising CiviCRM from f4db7fdb31 to 6936aa0262 |
[production] |
| 22:37 |
<bstorm_> |
Deleting the massive number of test ingresses for tool-fourohfour so the ingress controllers aren't moving so slowly. |
[toolsbeta] |
| 22:19 |
<bstorm_> |
Changed the ingress-admission ValidatingWebhookConfiguration to check extensions as well as networking API groups |
[toolsbeta] |
| 20:48 |
<ejegg> |
updated Fundraising CiviCRM from abf0019c44 to f4db7fdb31 |
[production] |