|
2020-08-28
ยง
|
| 16:33 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.08-restore-ttl |
[production] |
| 16:29 |
<rzl@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.07-set-readwrite (exit_code=0) |
[production] |
| 16:29 |
<rzl@cumin1001> |
[DRY-RUN] MediaWiki read-only period ends at: 2020-08-28 16:29:24.432463 |
[production] |
| 16:29 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.07-set-readwrite |
[production] |
| 16:29 |
<rzl@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite (exit_code=0) |
[production] |
| 16:29 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite |
[production] |
| 16:29 |
<rzl@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.05-invert-redis-sessions (exit_code=0) |
[production] |
| 16:29 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.05-invert-redis-sessions |
[production] |
| 16:29 |
<rzl@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki (exit_code=0) |
[production] |
| 16:29 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki |
[production] |
| 16:28 |
<rzl@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.03-set-db-readonly (exit_code=0) |
[production] |
| 16:28 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.03-set-db-readonly |
[production] |
| 16:28 |
<rzl@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.02-set-readonly (exit_code=0) |
[production] |
| 16:28 |
<rzl@cumin1001> |
[DRY-RUN] MediaWiki read-only period starts at: 2020-08-28 16:28:07.882663 |
[production] |
| 16:28 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.02-set-readonly |
[production] |
| 16:19 |
<rzl@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.01-stop-maintenance (exit_code=0) |
[production] |
| 16:19 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.01-stop-maintenance |
[production] |
| 16:13 |
<rzl@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.00-warmup-caches (exit_code=0) |
[production] |
| 16:12 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.00-warmup-caches |
[production] |
| 16:09 |
<rzl@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.00-reduce-ttl (exit_code=0) |
[production] |
| 16:09 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.00-reduce-ttl |
[production] |
| 16:08 |
<rzl@cumin1001> |
END (FAIL) - Cookbook sre.switchdc.mediawiki.00-reduce-ttl (exit_code=99) |
[production] |
| 16:08 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.00-reduce-ttl |
[production] |
| 16:07 |
<rzl@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.00-disable-puppet (exit_code=0) |
[production] |
| 16:07 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.00-disable-puppet |
[production] |
| 16:06 |
<rzl> |
starting one more live test of the data center switchover automation, no production impact is expected but there will be some SAL noise |
[production] |
| 14:22 |
<moritzm> |
installing Java security updates on kafka/main and Logstash(5) clusters |
[production] |
| 13:35 |
<hashar@deploy1001> |
Finished deploy [integration/docroot@65ec92c]: noop, sync up for README.md (duration: 00m 07s) |
[production] |
| 13:35 |
<hashar@deploy1001> |
Started deploy [integration/docroot@65ec92c]: noop, sync up for README.md |
[production] |
| 13:25 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:23 |
<elukey@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:07 |
<elukey> |
stop kafka on kafka-jumbo1006 and reimage to buster |
[production] |
| 12:56 |
<moritzm> |
installing debmonitor1002 T261492 |
[production] |
| 12:46 |
<moritzm> |
installing debmonitor2002 T261492 |
[production] |
| 11:50 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |
| 11:40 |
<jmm@cumin2001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 11:37 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |
| 11:27 |
<jmm@cumin2001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 11:27 |
<jmm@cumin2001> |
END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) |
[production] |
| 11:27 |
<jmm@cumin2001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 09:48 |
<jayme> |
updated helm to 2.16.9-3 on chartmuseum*, contint*, deploy* |
[production] |
| 09:19 |
<jayme> |
imported helm_2.16.9-3 to buster-wikimedia, stretch-wikimedia, jessie-wikimedia |
[production] |
| 08:22 |
<kormat> |
enabling replication from db2112 to db1083 (s1) T243373 |
[production] |
| 07:40 |
<jynus> |
restart backup2001,backup1002 |
[production] |
| 07:10 |
<jynus> |
restart db2139 |
[production] |
| 07:07 |
<marostegui> |
Warm up parsercache in codfw - T260042 |
[production] |
| 06:47 |
<jynus> |
restart db2102 |
[production] |
| 06:28 |
<jynus> |
restart db2100 |
[production] |
| 06:07 |
<jynus> |
restart db2099 |
[production] |
| 05:50 |
<jynus> |
restart db2098 |
[production] |