|
2019-05-30
ยง
|
| 13:17 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=restbase1014.eqiad.wmnet |
[production] |
| 13:17 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=restbase1013.eqiad.wmnet |
[production] |
| 13:17 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=restbase1012.eqiad.wmnet |
[production] |
| 13:17 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=restbase1011.eqiad.wmnet |
[production] |
| 13:16 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=restbase1010.eqiad.wmnet |
[production] |
| 13:16 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=restbase1009.eqiad.wmnet |
[production] |
| 13:16 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=restbase1008.eqiad.wmnet |
[production] |
| 13:16 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=restbase1007.eqiad.wmnet |
[production] |
| 13:08 |
<bblack> |
cp3047 puppet-disable + depool for reimage to ATS - T222937 |
[production] |
| 13:03 |
<marostegui> |
Stop MySQL on db1099 for onsite maintenance - T221502 |
[production] |
| 13:02 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1099 T221502 (duration: 00m 56s) |
[production] |
| 13:01 |
<arturo> |
uncordon/repool tools-worker-1001/2/3. They should be fine now. I'm only leaving 1029 cordoned for testing purposes |
[tools] |
| 13:01 |
<arturo> |
reboot tools-woker-1003 to cleanup sssd config and let nslcd/nscd start freshly |
[tools] |
| 13:00 |
<reedy@deploy1001> |
Synchronized php-1.34.0-wmf.7/tests/phpunit/includes/: T222628 (duration: 01m 06s) |
[production] |
| 12:58 |
<reedy@deploy1001> |
Synchronized php-1.34.0-wmf.7/includes/Linker.php: T222628 (duration: 01m 04s) |
[production] |
| 12:49 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:49 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:47 |
<arturo> |
reboot tools-woker-1002 to cleanup sssd config and let nslcd/nscd start freshly |
[tools] |
| 12:42 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:42 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:42 |
<arturo> |
reboot tools-woker-1001 to cleanup sssd config and let nslcd/nscd start freshly |
[tools] |
| 12:37 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:37 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:35 |
<arturo> |
enable puppet in tools-worker nodes |
[tools] |
| 12:29 |
<arturo> |
switch hiera setting back to classic/sudoldap for tools-worker because T224651 (T224558) |
[tools] |
| 12:25 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:25 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:25 |
<arturo> |
cordon/drain tools-worker-1002 because T224651 and T224651 |
[tools] |
| 12:23 |
<arturo> |
cordon/drain tools-worker-1001 because T224651 and T224651 |
[tools] |
| 12:22 |
<arturo> |
cordon/drain tools-worker-1029 because T224651 and T224651 |
[tools] |
| 12:21 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:21 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:20 |
<arturo> |
cordon/drain tools-worker-1003 because T224651 and T224651 |
[tools] |
| 12:15 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:15 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:08 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:08 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:08 |
<wm-bot> |
<jeanfred> Nuked the virtualenv and reinstalled all deps from scratch, in desperation for T224651 |
[tools.integraality] |
| 12:03 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:02 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 11:59 |
<arturo> |
T224558 repool tools-worker-1003 (using sssd/sudo now!) |
[tools] |
| 11:57 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 11:57 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 11:52 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 11:52 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 11:44 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 11:44 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 11:39 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 11:39 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 11:36 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |