|
2018-03-08
ยง
|
| 15:56 |
<andrewbogott> |
restarting nova-fullstack on labnet1001 |
[production] |
| 15:54 |
<andrewbogott> |
restarting nodepool again |
[production] |
| 15:42 |
<andrewbogott> |
stopping nodepool again because something isn't quite right |
[production] |
| 15:41 |
<marostegui> |
Power off es1019 - T187530 |
[production] |
| 15:32 |
<otto@tin> |
Synchronized wmf-config/ProductionServices.php: Point Mediawiki Monolog at new Kafka jumbo-eqiad cluster: T188136 (duration: 01m 16s) |
[production] |
| 15:30 |
<marostegui> |
Set m5 master db1009 read only for the failover - T189005 |
[production] |
| 15:29 |
<ottomata> |
merging and then deploying mediawiki-config to point monolog avro kafka producer at new kafka jumbo cluster: https://phabricator.wikimedia.org/T188136 |
[production] |
| 15:29 |
<andrewbogott> |
disabling puppet on labnodepool1001 |
[production] |
| 15:17 |
<andrewbogott> |
silencing nova and other openstack alerts in anticipation of service interruptions for https://phabricator.wikimedia.org/T189005 |
[production] |
| 15:01 |
<marostegui> |
Disable puppet on db1073 - T189005 |
[production] |
| 15:00 |
<marostegui> |
Change topology in m5, db2037 to become a slave of db1073 - T189005 |
[production] |
| 14:56 |
<oblivian@tin> |
Synchronized wmf-config/CommonSettings.php: Use EtcdConfig everywhere (duration: 01m 15s) |
[production] |
| 14:38 |
<zeljkof> |
EU SWAT finished |
[production] |
| 14:38 |
<marostegui> |
Stop mysql on es1019 - T187530 |
[production] |
| 14:37 |
<zfilipin@tin> |
Synchronized php-1.31.0-wmf.24/extensions/VisualEditor/modules/ve-mw/init/ve.init.mw.Target.js: SWAT: [[gerrit:417177|Blacklist Web of Trust junk from being added to pages (T189148)]] (duration: 01m 15s) |
[production] |
| 14:35 |
<zfilipin@tin> |
Synchronized php-1.31.0-wmf.24/extensions/VisualEditor/modules/ve-mw/init/ve.init.mw.ArticleTarget.js: SWAT: [[gerrit:417023|Follow-up I5357a909: Fix logic for autosave from edited state (T189071)]] (duration: 01m 16s) |
[production] |
| 14:28 |
<mobrovac@tin> |
Finished deploy [cpjobqueue/deploy@4fa1cf0]: Lower the refreshLinks concurrency to 175 - T185052 (duration: 00m 33s) |
[production] |
| 14:27 |
<mobrovac@tin> |
Started deploy [cpjobqueue/deploy@4fa1cf0]: Lower the refreshLinks concurrency to 175 - T185052 |
[production] |
| 14:26 |
<vgutierrez> |
uploaded pybal_1.15.2_all.deb to apt.wikimedia.org jessie-wikimedia |
[production] |
| 14:26 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:413653|2017 wikitext editor: Enable by default on officewiki (T188028)]] (duration: 01m 16s) |
[production] |
| 14:15 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:417257|Create the rollbacker group at ar.wikinews (T189206)]] (duration: 01m 16s) |
[production] |
| 13:56 |
<gehel> |
restart wdqs-updater on wdqs1005 to validate new config option - T188716 |
[production] |
| 13:52 |
<sbisson@tin> |
Finished deploy [kartotherian/deploy@42b3280]: Deploying kartotherian with updated dependencies and zoom lovel 19 to test servers (duration: 08m 31s) |
[production] |
| 13:44 |
<moritzm> |
depooling mwdebug2001, the host will temporarily be using an HHVM build linked against libicu57 to perform some tests |
[production] |
| 13:43 |
<sbisson@tin> |
Started deploy [kartotherian/deploy@42b3280]: Deploying kartotherian with updated dependencies and zoom lovel 19 to test servers |
[production] |
| 13:40 |
<elukey> |
eventlogging analytics migrated from eventlog1001 to eventlog1002 |
[production] |
| 13:35 |
<ariel@tin> |
Finished deploy [dumps/dumps@f26c114]: fix prefetch stubs; retrieval globals more robustly (duration: 00m 03s) |
[production] |
| 13:35 |
<ariel@tin> |
Started deploy [dumps/dumps@f26c114]: fix prefetch stubs; retrieval globals more robustly |
[production] |
| 13:29 |
<ema> |
cp-ulsfo: reboot for retpoline kernel updates T188092 |
[production] |
| 12:50 |
<oblivian@puppetmaster1001> |
conftool action : edit; selector: scope=codfw,name=ReadOnly |
[production] |
| 12:47 |
<oblivian@puppetmaster1001> |
conftool action : edit; selector: scope=codfw,name=ReadOnly |
[production] |
| 12:06 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1114 fully (duration: 01m 16s) |
[production] |
| 11:32 |
<moritzm> |
installing isc-dhcp security updates |
[production] |
| 10:43 |
<moritzm> |
installing libvpx security updates |
[production] |
| 10:22 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Change db1114 load (duration: 01m 16s) |
[production] |
| 10:14 |
<akosiaris> |
conduct IO stresstests on ganeti1005 (sca1004 VM) with cache=none KVM flag on T181121 |
[production] |
| 10:13 |
<akosiaris> |
conduct IO stresstests on ganeti1005 (sca1004 VM) with cache=none KVM flag on |
[production] |
| 09:57 |
<dcausse> |
restaring mjolnir-kafka-daemon.service on relforge1002 to switch to kafka jumbo |
[production] |
| 09:56 |
<dcausse> |
restaring mjolnir-kafka-daemon.service on relforge1001 to switch to kafka jumbo |
[production] |
| 09:56 |
<_joe_> |
decommissioning mw2017-2099 T187467 |
[production] |
| 09:46 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1114 partially (duration: 01m 16s) |
[production] |
| 09:44 |
<moritzm> |
rearming keyholder on neodymium after reboot |
[production] |
| 09:40 |
<moritzm> |
rebooting neodymium for kernel security update |
[production] |
| 09:22 |
<ema> |
cp-eqsin: reboot for retpoline kernel updates T188092 |
[production] |
| 09:12 |
<ema> |
cp3043: varnish-be-restart T189085 |
[production] |
| 09:08 |
<moritzm> |
rebooting bast1001 for kernel security update |
[production] |
| 08:58 |
<elukey> |
restart varnish backend on cp3041 (failed fetches) |
[production] |
| 08:58 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2046, db2053 and db2060 after kernel upgrade (duration: 01m 15s) |
[production] |
| 08:58 |
<moritzm> |
reset RAC on bast1001, serial console was stuck |
[production] |
| 08:50 |
<elukey> |
rebooting analytics1003 (Hadoop Hive, Oozie, etc..) for kernel updates |
[production] |