|
2018-04-17
ยง
|
| 19:01 |
<imarlier@tin> |
Started deploy [performance/navtiming@22483a4]: Navtiming refactor for increased testability, and to add wrapper for easy service use |
[production] |
| 18:52 |
<urandom> |
rebooting restbase-dev1006 (kernel oom killer misbehaving) |
[production] |
| 18:45 |
<urandom> |
rebooting restbase-dev1005 (kernel oom killer misbehaving) |
[production] |
| 18:41 |
<urandom> |
rebooting restbase-dev1004 (kernel oom killer misbehaving) |
[production] |
| 17:56 |
<demon@tin> |
Started scap: bootstrap wmf.30 |
[production] |
| 17:27 |
<ejegg> |
updated payments-wiki from 320a6c2600 to 4a8aada491 |
[production] |
| 17:16 |
<mobrovac@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 100% of anons for enwiki - T191101 (duration: 00m 59s) |
[production] |
| 16:57 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool es1017 fully (duration: 01m 16s) |
[production] |
| 16:37 |
<elukey> |
incremental rollout of the new zookeeper jmx config to druid1* and conf* |
[production] |
| 16:34 |
<urandom> |
decommissioning Cassandra, restbase1010-a -- T189822 |
[production] |
| 16:02 |
<mobrovac@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 75% of anons for enwiki - T191101 (duration: 00m 58s) |
[production] |
| 15:50 |
<arturo> |
enable puppet in labstore1004 |
[production] |
| 15:37 |
<vgutierrez> |
Repool (Enable BGP) on lvs3004 - T191897 |
[production] |
| 15:33 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Change db2048 IP - T191193 (duration: 00m 58s) |
[production] |
| 15:32 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Change db2048 IP - T191193 (duration: 00m 58s) |
[production] |
| 15:23 |
<marostegui> |
Stopping mysql on db2048 will break replication on codfw s1 slaves |
[production] |
| 15:23 |
<marostegui> |
Stop MySQL on db2048 for rack movement - T191193 |
[production] |
| 15:22 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1067, es1017 with low load (duration: 01m 02s) |
[production] |
| 14:59 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1114 after changing the network cable - T191996 (duration: 01m 02s) |
[production] |
| 14:55 |
<gehel> |
starting data reimport after re-image for wdqs2001 - T189192 |
[production] |
| 14:52 |
<marostegui> |
Stop MySQL on db2042 to move it to another rack - https://phabricator.wikimedia.org/T191193 |
[production] |
| 14:36 |
<ariel@tin> |
Finished deploy [dumps/dumps@1073d75]: more exception logging from xmlstream (duration: 00m 03s) |
[production] |
| 14:36 |
<ariel@tin> |
Started deploy [dumps/dumps@1073d75]: more exception logging from xmlstream |
[production] |
| 14:30 |
<mobrovac@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 50% of anons for enwiki - T191101 (duration: 00m 58s) |
[production] |
| 14:25 |
<mobrovac@tin> |
Synchronized php-1.31.0-wmf.29/extensions/EventBus/includes/JobQueueEventBus.php: Support per-event dispatch of events, file 3/3 - T191464 (duration: 03m 07s) |
[production] |
| 14:23 |
<jynus> |
start es1017 reimage |
[production] |
| 14:22 |
<mobrovac@tin> |
Synchronized php-1.31.0-wmf.29/extensions/EventBus/includes/EventBus.php: Support per-event dispatch of events, file 2/3 - T191464 (duration: 03m 06s) |
[production] |
| 14:16 |
<mobrovac@tin> |
Synchronized php-1.31.0-wmf.29/extensions/EventBus/extension.json: Support per-event dispatch of events, file 1/3 - T191464 (duration: 03m 00s) |
[production] |
| 14:08 |
<vgutierrez> |
Depool and reimage lvs3004 as stretch - T191897 |
[production] |
| 13:42 |
<mobrovac@tin> |
Synchronized php-1.31.0-wmf.29/extensions/EventBus/extension.json: Support per-event dispatch of events, file 1/3 - T191464 (duration: 03m 07s) |
[production] |
| 13:33 |
<moritzm> |
removed role::mediawiki::imagescaler from deployment-mediawiki05, per watroles the only use of that role in WMCS |
[production] |
| 13:32 |
<moritzm> |
removed role::mediawiki::imagescaler from deployment-prep, per watroles the only use of that role in WMCS |
[production] |
| 13:30 |
<jynus> |
starting backup from db1067, may generate some lag |
[production] |
| 13:26 |
<volans> |
updating puppet compiler facts |
[production] |
| 13:25 |
<elukey> |
completed migration of zookeeper on conf200[123] |
[production] |
| 13:15 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1067 (duration: 00m 58s) |
[production] |
| 13:06 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1114 to get it ready for a network cable change (duration: 00m 58s) |
[production] |
| 13:00 |
<elukey> |
upgrade zookeeper on conf200[123] to 3.4.9~jessie - T182924 |
[production] |
| 12:31 |
<mobrovac@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 25% of annons on enwiki, take #2 - T191101 (duration: 00m 58s) |
[production] |
| 12:04 |
<mobrovac@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable page previews for 25% of annons on enwiki - T191101 (duration: 01m 03s) |
[production] |
| 10:52 |
<ema> |
lvs100[63] restart pybal to apply https://gerrit.wikimedia.org/r/424553 T188062 |
[production] |
| 10:39 |
<ema> |
lvs200[63]: restart pybal to apply https://gerrit.wikimedia.org/r/424553 T188062 |
[production] |
| 10:03 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@e463fcf]: Use keep-alive for connections to AQS (duration: 20m 17s) |
[production] |
| 09:43 |
<mobrovac@tin> |
Started deploy [restbase/deploy@e463fcf]: Use keep-alive for connections to AQS |
[production] |
| 09:37 |
<moritzm> |
reimaging mw1280, mw1281, mw1282 (API servers) to stretch |
[production] |
| 09:36 |
<moritzm> |
reimaging mw1266, mw1267, mw1268 (app servers) to stretch |
[production] |
| 09:17 |
<godog> |
restart xenon-log on mwlog* - T169249 |
[production] |
| 08:46 |
<vgutierrez@neodymium> |
conftool action : set/pooled=yes; selector: name=chromium.wikimedia.org,service=pdns_recursor |
[production] |
| 08:19 |
<elukey> |
restart nrpe-server on kafka2001 (kafka check not defined) |
[production] |
| 08:01 |
<moritzm> |
rolling restart of HHVM on video scalers to pick up ICU security update |
[production] |