|
2020-08-12
ยง
|
| 20:13 |
<root@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 20:11 |
<root@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 20:11 |
<root@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 20:11 |
<wkandek> |
reboot mw1272 |
[production] |
| 20:11 |
<wkandek> |
reboot mw1271 |
[production] |
| 19:41 |
<hashar> |
Upgrading Jenkins on contint2001 (primary) |
[production] |
| 19:25 |
<hashar> |
contint1001: sudo systemctl mask jenkins # spare server |
[production] |
| 19:25 |
<mutante> |
all releases* servers except 1001 - disable puppet; stop jenkins, mask jenkins |
[production] |
| 19:22 |
<mutante> |
releases1002 - stopped and masked jenkins service |
[production] |
| 19:22 |
<mutante> |
releases2001 - stopped and masked jenkins service |
[production] |
| 19:20 |
<mutante> |
upgrading jenkins on releases*001 |
[production] |
| 19:19 |
<hashar> |
Upgrading Jenkins on contint1001 (spare) |
[production] |
| 19:16 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.36.0-wmf.4 |
[production] |
| 19:13 |
<mutante> |
uploade new jenkins version to APT repo; upgrading jenkins on releases1002/2002 |
[production] |
| 19:08 |
<effie> |
pool mw1396 |
[production] |
| 19:06 |
<effie> |
repool mw1395 mw1397 mw1399 |
[production] |
| 18:56 |
<root@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 18:55 |
<root@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 18:50 |
<ladsgroup@deploy1001> |
Synchronized php-1.36.0-wmf.4/extensions/Wikibase/client/includes/Store/Sql/DirectSqlStore.php: [[phab:T255305|Set caching of CachingEntityRevisionLookup to CACHE_NONE in client]] (duration: 02m 13s) |
[production] |
| 18:47 |
<wkandek> |
reboot mw1270 |
[production] |
| 18:47 |
<root@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 18:45 |
<wkandek> |
reboot mw1269 |
[production] |
| 18:41 |
<root@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 18:39 |
<jiji@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) |
[production] |
| 18:39 |
<jiji@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) |
[production] |
| 18:38 |
<jiji@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) |
[production] |
| 18:28 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 18:25 |
<wkandek> |
reboot mw1268 |
[production] |
| 18:25 |
<otto@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'eventgate-main' for release 'production' . |
[production] |
| 18:25 |
<otto@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'eventgate-main' for release 'canary' . |
[production] |
| 18:23 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 18:22 |
<otto@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'eventgate-main' for release 'canary' . |
[production] |
| 18:22 |
<otto@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'eventgate-main' for release 'production' . |
[production] |
| 18:22 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 18:22 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 18:22 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 18:17 |
<jiji@cumin1001> |
END (ERROR) - Cookbook sre.hosts.reboot-single (exit_code=97) |
[production] |
| 18:17 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 18:16 |
<jiji@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) |
[production] |
| 18:10 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable GrowthExperiments on hewiki (T255020) (duration: 01m 03s) |
[production] |
| 18:08 |
<jiji@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) |
[production] |
| 18:07 |
<otto@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'eventgate-analytics-external' for release 'canary' . |
[production] |
| 18:07 |
<otto@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'eventgate-analytics-external' for release 'production' . |
[production] |
| 18:06 |
<jiji@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) |
[production] |
| 18:04 |
<ladsgroup@deploy1001> |
Synchronized php-1.36.0-wmf.4/extensions/Wikibase/repo/includes/Store/Sql/SqlStore.php: [[phab:T255305|Set caching of CachingEntityRevisionLookup to CACHE_NONE in repo]] (duration: 01m 06s) |
[production] |
| 18:02 |
<otto@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'eventgate-logging-external' for release 'canary' . |
[production] |
| 18:02 |
<otto@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'eventgate-logging-external' for release 'production' . |
[production] |
| 18:00 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 17:59 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 17:58 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |