|
2018-04-17
ยง
|
| 15:23 |
<marostegui> |
Stopping mysql on db2048 will break replication on codfw s1 slaves |
[production] |
| 15:23 |
<marostegui> |
Stop MySQL on db2048 for rack movement - T191193 |
[production] |
| 15:22 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1067, es1017 with low load (duration: 01m 02s) |
[production] |
| 14:59 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1114 after changing the network cable - T191996 (duration: 01m 02s) |
[production] |
| 14:55 |
<gehel> |
starting data reimport after re-image for wdqs2001 - T189192 |
[production] |
| 14:52 |
<marostegui> |
Stop MySQL on db2042 to move it to another rack - https://phabricator.wikimedia.org/T191193 |
[production] |
| 14:36 |
<ariel@tin> |
Finished deploy [dumps/dumps@1073d75]: more exception logging from xmlstream (duration: 00m 03s) |
[production] |
| 14:36 |
<ariel@tin> |
Started deploy [dumps/dumps@1073d75]: more exception logging from xmlstream |
[production] |
| 14:30 |
<mobrovac@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 50% of anons for enwiki - T191101 (duration: 00m 58s) |
[production] |
| 14:25 |
<mobrovac@tin> |
Synchronized php-1.31.0-wmf.29/extensions/EventBus/includes/JobQueueEventBus.php: Support per-event dispatch of events, file 3/3 - T191464 (duration: 03m 07s) |
[production] |
| 14:23 |
<jynus> |
start es1017 reimage |
[production] |
| 14:22 |
<mobrovac@tin> |
Synchronized php-1.31.0-wmf.29/extensions/EventBus/includes/EventBus.php: Support per-event dispatch of events, file 2/3 - T191464 (duration: 03m 06s) |
[production] |
| 14:16 |
<mobrovac@tin> |
Synchronized php-1.31.0-wmf.29/extensions/EventBus/extension.json: Support per-event dispatch of events, file 1/3 - T191464 (duration: 03m 00s) |
[production] |
| 14:08 |
<vgutierrez> |
Depool and reimage lvs3004 as stretch - T191897 |
[production] |
| 13:42 |
<mobrovac@tin> |
Synchronized php-1.31.0-wmf.29/extensions/EventBus/extension.json: Support per-event dispatch of events, file 1/3 - T191464 (duration: 03m 07s) |
[production] |
| 13:33 |
<moritzm> |
removed role::mediawiki::imagescaler from deployment-mediawiki05, per watroles the only use of that role in WMCS |
[production] |
| 13:32 |
<moritzm> |
removed role::mediawiki::imagescaler from deployment-prep, per watroles the only use of that role in WMCS |
[production] |
| 13:30 |
<jynus> |
starting backup from db1067, may generate some lag |
[production] |
| 13:26 |
<volans> |
updating puppet compiler facts |
[production] |
| 13:25 |
<elukey> |
completed migration of zookeeper on conf200[123] |
[production] |
| 13:15 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1067 (duration: 00m 58s) |
[production] |
| 13:06 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1114 to get it ready for a network cable change (duration: 00m 58s) |
[production] |
| 13:00 |
<elukey> |
upgrade zookeeper on conf200[123] to 3.4.9~jessie - T182924 |
[production] |
| 12:31 |
<mobrovac@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 25% of annons on enwiki, take #2 - T191101 (duration: 00m 58s) |
[production] |
| 12:04 |
<mobrovac@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 25% of annons on enwiki - T191101 (duration: 01m 03s) |
[production] |
| 10:52 |
<ema> |
lvs100[63] restart pybal to apply https://gerrit.wikimedia.org/r/424553 T188062 |
[production] |
| 10:39 |
<ema> |
lvs200[63]: restart pybal to apply https://gerrit.wikimedia.org/r/424553 T188062 |
[production] |
| 10:03 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@e463fcf]: Use keep-alive for connections to AQS (duration: 20m 17s) |
[production] |
| 09:43 |
<mobrovac@tin> |
Started deploy [restbase/deploy@e463fcf]: Use keep-alive for connections to AQS |
[production] |
| 09:37 |
<moritzm> |
reimaging mw1280, mw1281, mw1282 (API servers) to stretch |
[production] |
| 09:36 |
<moritzm> |
reimaging mw1266, mw1267, mw1268 (app servers) to stretch |
[production] |
| 09:17 |
<godog> |
restart xenon-log on mwlog* - T169249 |
[production] |
| 08:46 |
<vgutierrez@neodymium> |
conftool action : set/pooled=yes; selector: name=chromium.wikimedia.org,service=pdns_recursor |
[production] |
| 08:19 |
<elukey> |
restart nrpe-server on kafka2001 (kafka check not defined) |
[production] |
| 08:01 |
<moritzm> |
rolling restart of HHVM on video scalers to pick up ICU security update |
[production] |
| 07:42 |
<moritzm> |
installing ICU security updates |
[production] |
| 07:27 |
<jynus> |
restarting dbstore2001 |
[production] |
| 07:14 |
<moritzm> |
installing perl security updates on trusty |
[production] |
| 06:48 |
<vgutierrez@neodymium> |
conftool action : set/pooled=no; selector: name=chromium.wikimedia.org,service=pdns_recursor |
[production] |
| 06:47 |
<vgutierrez> |
Depool and reimage chromium as stretch - T187090 |
[production] |
| 06:03 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Fully repool db1114 in API - T191996 (duration: 00m 58s) |
[production] |
| 05:52 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Give more API traffic to db1114 (duration: 00m 58s) |
[production] |
| 05:43 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Slowly repool db1114 in API (duration: 00m 58s) |
[production] |
| 05:35 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Restore main traffic original weight for db1114 (duration: 00m 58s) |
[production] |
| 05:24 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Slowly repool db1114 (duration: 00m 58s) |
[production] |
| 05:21 |
<marostegui> |
Deploy schema change on db1092 - T187089 T185128 T153182 |
[production] |
| 05:19 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1092 for alter table (duration: 00m 58s) |
[production] |
| 05:11 |
<marostegui> |
Stop MySQL and reboot db1114 to boot up with the new kernel |
[production] |
| 05:10 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1114 (duration: 00m 59s) |
[production] |
| 02:32 |
<l10nupdate@tin> |
scap sync-l10n completed (1.31.0-wmf.29) (duration: 05m 27s) |
[production] |