|
2020-08-03
§
|
| 08:07 |
<elukey> |
roll restart aqs on aqs* to pick up new druid settings |
[production] |
| 07:10 |
<marostegui> |
Remove revision triggers from db2095:3317 for MCR changes T238966 |
[production] |
| 07:09 |
<marostegui> |
Deploy MCR change on s7 codfw, lag will appear on codfw T238966 |
[production] |
| 07:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1106 after compression', diff saved to https://phabricator.wikimedia.org/P12139 and previous config saved to /var/cache/conftool/dbconfig/20200803-070702-marostegui.json |
[production] |
| 05:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1106 after compression', diff saved to https://phabricator.wikimedia.org/P12138 and previous config saved to /var/cache/conftool/dbconfig/20200803-052715-marostegui.json |
[production] |
| 05:04 |
<marostegui> |
Remove db1108:3321 and db1108:3322 from tendril and add db1108:3351 and db1108:3352 T254462 |
[production] |
| 05:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1106 after compression', diff saved to https://phabricator.wikimedia.org/P12137 and previous config saved to /var/cache/conftool/dbconfig/20200803-050148-marostegui.json |
[production] |
|
2020-07-31
§
|
| 23:48 |
<ejegg> |
updated payments-wiki from c365c136d2 to cd012f37f1 |
[production] |
| 22:53 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 22:53 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:53 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 22:53 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:03 |
<mutante> |
wtp2019 - parsoid could not start after reimaging - was missing /etc/parsoid/config.yaml which is a symbolic link deep onto /srv/deployment/parsoid/deploy-cache/.. like in some other cases before manually deleted deploy-cache dir and ran puppet again .. T258775 |
[production] |
| 21:58 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=wtp2020.codfw.wmnet |
[production] |
| 21:57 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=wtp2019.codfw.wmnet |
[production] |
| 21:50 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=wtp2019.codfw.wmnet |
[production] |
| 21:36 |
<mutante> |
[wtp2019:~] $ sudo rm -rf /srv/deployment/parsoid/deploy-cache |
[production] |
| 21:13 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=wtp2018.codfw.wmnet |
[production] |
| 20:57 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 20:56 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 20:56 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 20:56 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 20:56 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 20:56 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 20:55 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 20:55 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:55 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:55 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:55 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:55 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:55 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:55 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:22 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 20:22 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:22 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 20:22 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:11 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 20:11 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:03 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 20:02 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 19:51 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 19:51 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 19:39 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 19:39 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 19:20 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=wtp2017.codfw.wmnet |
[production] |
| 18:57 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=wtp2016.codfw.wmnet |
[production] |
| 17:45 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 17:45 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |