|
2020-03-30
§
|
| 08:10 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 07:53 |
<vgutierrez> |
depool & decommission cp2002 - T248818 |
[production] |
| 07:48 |
<marostegui> |
Run cloudcontrol1003:~# wmcs-wikireplica-dns to promote dbproxy1018 to wikireplicas active proxy T231520 |
[production] |
| 07:40 |
<marostegui> |
Replace dbproxy1010 with dbproxy1011 for wiki replicas, analytics - T231520 |
[production] |
| 07:28 |
<marostegui> |
Deploy schema change on labswiki (wikitech) - T248333 |
[production] |
| 07:26 |
<marostegui> |
Deploy schema change on s4 codfw, this will generate lag on codfw - T248333 |
[production] |
| 07:17 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 07:17 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 07:10 |
<vgutierrez> |
depool and decommission cp2001 - T248815 |
[production] |
| 06:52 |
<vgutierrez> |
pool cp2028 - T247340 |
[production] |
| 06:29 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 06:28 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1074 after schema change', diff saved to https://phabricator.wikimedia.org/P10813 and previous config saved to /var/cache/conftool/dbconfig/20200330-062858-marostegui.json |
[production] |
| 06:26 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 06:04 |
<marostegui> |
Deploy schema change on db1074 with replication, this will generate lag on s2 labs |
[production] |
| 06:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1074 for schema change', diff saved to https://phabricator.wikimedia.org/P10812 and previous config saved to /var/cache/conftool/dbconfig/20200330-060338-marostegui.json |
[production] |
| 05:40 |
<vgutierrez> |
pool cp2027 - T247340 |
[production] |
| 05:13 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 05:10 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:55 |
<vgutierrez> |
Enable TLS Session tickets in ulsfo - T245616 |
[production] |
| 04:32 |
<vgutierrez> |
upgrade ATS to version 8.0.6-1wm4 on ulsfo - T245616 |
[production] |
|
2020-03-27
§
|
| 20:51 |
<ejegg> |
updated payments-wiki from db618f429d to 1640f5e21e |
[production] |
| 15:15 |
<andrew@deploy1001> |
Finished deploy [horizon/deploy@33e67f9]: fix Identity->Projects with keystone Queens (duration: 03m 35s) |
[production] |
| 15:12 |
<andrew@deploy1001> |
Started deploy [horizon/deploy@33e67f9]: fix Identity->Projects with keystone Queens |
[production] |
| 14:41 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1129 after schema change', diff saved to https://phabricator.wikimedia.org/P10807 and previous config saved to /var/cache/conftool/dbconfig/20200327-144125-marostegui.json |
[production] |
| 14:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1129 for schema change', diff saved to https://phabricator.wikimedia.org/P10806 and previous config saved to /var/cache/conftool/dbconfig/20200327-142240-marostegui.json |
[production] |
| 14:19 |
<moritzm> |
updating linux-image-4.9.0-11-amd64 where applicable |
[production] |
| 13:30 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1076 after schema change', diff saved to https://phabricator.wikimedia.org/P10805 and previous config saved to /var/cache/conftool/dbconfig/20200327-133022-marostegui.json |
[production] |
| 13:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1076 for schema change', diff saved to https://phabricator.wikimedia.org/P10804 and previous config saved to /var/cache/conftool/dbconfig/20200327-130706-marostegui.json |
[production] |
| 13:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1105:3312 after schema change', diff saved to https://phabricator.wikimedia.org/P10803 and previous config saved to /var/cache/conftool/dbconfig/20200327-130542-marostegui.json |
[production] |
| 12:49 |
<Amir1> |
ladsgroup@mwmaint1002:~$ mwscript createAndPromote.php --wiki=labswiki --force "Ladsgroup" --interface-admin |
[production] |
| 12:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1105:3312 for schema change', diff saved to https://phabricator.wikimedia.org/P10802 and previous config saved to /var/cache/conftool/dbconfig/20200327-122144-marostegui.json |
[production] |
| 12:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1103:3312 after schema change', diff saved to https://phabricator.wikimedia.org/P10801 and previous config saved to /var/cache/conftool/dbconfig/20200327-122058-marostegui.json |
[production] |
| 12:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1103:3312 for schema change', diff saved to https://phabricator.wikimedia.org/P10800 and previous config saved to /var/cache/conftool/dbconfig/20200327-120234-marostegui.json |
[production] |
| 11:54 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: dc=codfw,cluster=restbase,service=restbase-backend,name=restbase202[123].codfw.wmnet |
[production] |
| 11:51 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: dc=codfw,cluster=restbase,service=restbase-ssl,name=restbase202[123].codfw.wmnet |
[production] |
| 11:45 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: dc=codfw,cluster=restbase,service=restbase,name=restbase2023.codfw.wmnet |
[production] |
| 11:44 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: dc=codfw,cluster=restbase,service=restbase,name=restbase2022.codfw.wmnet |
[production] |
| 11:44 |
<oblivian@puppetmaster1001> |
conftool action : edit; selector: dc=codfw,cluster=restbase,service=restbase-ssl,name=restbase202[1].codfw.wmnet |
[production] |
| 11:44 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: dc=codfw,cluster=restbase,service=restbase,name=restbase2021.codfw.wmnet |
[production] |
| 10:55 |
<mutante> |
revoke puppet cert webserver-misc-apps.discovery.wmnet and recreate with additional SANs for new VMs |
[production] |
| 10:45 |
<mutante> |
miscweb1002 - upload and unpack RackTables-0.21.4 (T247646 T247648) |
[production] |
| 10:28 |
<marostegui> |
Alter db2125 s2 to set page_restrictions to default NULL - T248333 |
[production] |
| 10:12 |
<mutante> |
miscweb1002 - sed -i 's/tin.eqiad/deployment.eqiad/g' /srv/deployment/iegreview/iegreview-cache/.config T247648 |
[production] |
| 10:04 |
<vgutierrez> |
upload trafficserver 8.0.6-1wm4 to apt.wm.o (buster) - T245616 T170567 |
[production] |