|
2020-07-15
§
|
| 06:00 |
<marostegui> |
Starting x1 failover from db1120 to db1103 - T254871 |
[production] |
| 05:29 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1099:3318 ', diff saved to https://phabricator.wikimedia.org/P11909 and previous config saved to /var/cache/conftool/dbconfig/20200715-052939-marostegui.json |
[production] |
| 04:46 |
<marostegui> |
Start x1 pre failover steps T254871 |
[production] |
| 04:44 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Set db1103 weight to 0 before the switchover T254871', diff saved to https://phabricator.wikimedia.org/P11908 and previous config saved to /var/cache/conftool/dbconfig/20200715-044432-marostegui.json |
[production] |
| 04:43 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1135', diff saved to https://phabricator.wikimedia.org/P11907 and previous config saved to /var/cache/conftool/dbconfig/20200715-044332-marostegui.json |
[production] |
| 01:45 |
<eileen> |
tools revision changed from a9e7dc1559 to 7b6018a16e |
[production] |
| 00:26 |
<ryankemper@deploy1001> |
Finished deploy [wdqs/wdqs@8f6f660]: 0.3.41 (duration: 15m 10s) |
[production] |
| 00:11 |
<ryankemper@deploy1001> |
Started deploy [wdqs/wdqs@8f6f660]: 0.3.41 |
[production] |
|
2020-07-14
§
|
| 19:52 |
<jforrester@deploy1001> |
Synchronized php-1.35.0-wmf.41/vendor/wikimedia/parsoid/: T252448 T255190 Bump Parsoid to v0.12.0-a23 (duration: 01m 06s) |
[production] |
| 18:13 |
<ryankemper> |
all long-running elasticsearch reindex jobs are complete |
[production] |
| 18:09 |
<jforrester@deploy1001> |
Synchronized dblists/: T32405 T254287 Remove the mobilemainpagelegacy dblist (duration: 01m 04s) |
[production] |
| 18:07 |
<jforrester@deploy1001> |
Synchronized multiversion/MWConfigCacheGenerator.php: T32405 T254287 Stop loading the mobilemainpagelegacy dblist (duration: 01m 05s) |
[production] |
| 18:05 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T32405 T254287 Stop varying wgMFSpecialCaseMainPage (duration: 01m 05s) |
[production] |
| 15:56 |
<elukey> |
upgrade spark2 on stat100x to 2.4.4-bin-hadoop2.6-3 |
[production] |
| 15:40 |
<hnowlan@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'changeprop' for release 'production' . |
[production] |
| 15:25 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 15:23 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 15:17 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 15:13 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 15:12 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 15:10 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 15:04 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 15:02 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 14:57 |
<jforrester@deploy1001> |
Synchronized php-1.35.0-wmf.41/skins/Vector/includes/SkinVector.php: T257914 Restore div wrapper around print footer (duration: 01m 03s) |
[production] |
| 14:53 |
<hnowlan@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'changeprop' for release 'production' . |
[production] |
| 14:50 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 14:48 |
<jforrester@deploy1001> |
Synchronized php-1.35.0-wmf.41/extensions/WikibaseMediaInfo/src/WikibaseMediaInfoHooks.php: Fix case of directory name (duration: 01m 05s) |
[production] |
| 14:48 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 14:48 |
<moritzm> |
rebooting apt1001 for kernel update |
[production] |
| 14:42 |
<jynus> |
stopping db1117:3322 (m2) replication temp. for otrs db cloning T257928 |
[production] |
| 14:40 |
<hnowlan@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . |
[production] |
| 14:33 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 14:31 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 14:26 |
<oblivian@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'cxserver' for release 'production' . |
[production] |
| 14:23 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 14:21 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 14:20 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 14:18 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 14:18 |
<jmm@cumin2001> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) |
[production] |
| 14:18 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 14:14 |
<oblivian@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'cxserver' for release 'production' . |
[production] |
| 14:13 |
<andrewbogott> |
upgrading wikitech-static to mw 1.34.2 |
[production] |
| 14:11 |
<oblivian@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'cxserver' for release 'staging' . |
[production] |
| 14:05 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 14:01 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 13:42 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.hadoop.change-distro (exit_code=0) |
[production] |
| 13:28 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1112', diff saved to https://phabricator.wikimedia.org/P11900 and previous config saved to /var/cache/conftool/dbconfig/20200714-132823-marostegui.json |
[production] |
| 13:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1130', diff saved to https://phabricator.wikimedia.org/P11899 and previous config saved to /var/cache/conftool/dbconfig/20200714-132742-marostegui.json |
[production] |
| 13:27 |
<jbond@cumin1001> |
conftool action : set/pooled=yes; selector: name=dns1001.wikimedia.org |
[production] |
| 13:24 |
<jbond42> |
reboot dns1001 |
[production] |