2151-2200 of 10000 results (52ms)
2018-02-22 ยง
16:58 <XioNoX> redirecting ns2 traffic to radon [production]
16:49 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1076 (duration: 01m 12s) [production]
16:28 <ejegg> updated CiviCRM from b27e6a5019 to aa251f1a93 [production]
16:26 <mobrovac@tin> Synchronized wmf-config/jobqueue.php: Use EventBus for refreshLinks in test wikis, file 2/2 - T185052 (duration: 01m 12s) [production]
16:25 <mobrovac@tin> Synchronized wmf-config/InitialiseSettings.php: Use EventBus for refreshLinks in test wikis, file 1/2 - T185052 (duration: 01m 12s) [production]
16:23 <ppchelko@tin> Finished deploy [cpjobqueue/deploy@ab3d002]: Enable refreshLinks for group0 wikis T185052 (duration: 00m 36s) [production]
16:23 <ppchelko@tin> Started deploy [cpjobqueue/deploy@ab3d002]: Enable refreshLinks for group0 wikis T185052 [production]
16:22 <mobrovac@tin> scap failed: average error rate on 8/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) [production]
16:13 <jynus> tendril and dbtree database currently under maintanance [production]
16:04 <ejegg> updated payments-wiki from fe311c2d26 to 1acfc4a9a0 [production]
15:26 <ema> finished upgrading cache_text@ulsfo to varnish 5 [production]
15:24 <elukey> manually removing from cp1008 and cache::misc old files related to the varnishkafka jumbo testing instance (after https://gerrit.wikimedia.org/r/413370) [production]
14:58 <matthiasmullie> EU SWAT finished [production]
14:52 <mlitn@tin> Synchronized wmf-config/CommonSettings.php: Enable 3D file display (duration: 01m 12s) [production]
14:50 <mlitn@tin> Synchronized php-1.31.0-wmf.21/extensions/3D/extension.json: Remove MMV dependency for 3D (duration: 01m 12s) [production]
14:41 <ottomata> beginning migration of webrequest_misc from Kafka analytics to jumbo: T185136 [production]
14:40 <mlitn@tin> scap failed: average error rate on 3/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) [production]
14:38 <mlitn@tin> Synchronized wmf-config/InitialiseSettings.php: Enable 3D file display (duration: 01m 13s) [production]
14:32 <jmm@puppetmaster1001> conftool action : set/pooled=no; selector: name=mw2171.codfw.wmnet [production]
14:26 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:396318|Show HTML summaries on cswiki (T182321)]] (duration: 01m 13s) [production]
13:41 <ema> bounce pybal on lvs1003 to try establish missing etcd connections (zotero, thumbor, wdqs) https://phabricator.wikimedia.org/P6730 [production]
13:30 <moritzm> rebooting kubernetes1001 [production]
13:21 <ema> upgrade pybal on lvs1003 to 1.14.4 [production]
12:42 <_joe_> ended live-hacking on mwdebug1001 (T185078) [production]
12:24 <_joe_> live-hacking ProductionServices.php on mwdebug1001 for testing (T185078) [production]
11:50 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1089 and slowly repool db1076 (duration: 01m 12s) [production]
11:40 <kartik@tin> Finished deploy [cxserver/deploy@300f728]: Update cxserver to b0404d1 (duration: 03m 37s) [production]
11:39 <akosiaris> purge ORES from scb hosts T168073 T171851 [production]
11:37 <kartik@tin> Started deploy [cxserver/deploy@300f728]: Update cxserver to b0404d1 [production]
11:19 <_joe_> upgrading python-conftool on all cache hosts [production]
10:55 <ema> upgrading python-conftool on cp5007 [production]
10:51 <_joe_> upgrading python-conftool on cp1008 [production]
10:42 <jynus> stop db2073 for maintenance [production]
10:40 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1089 and fully repool db1104 (duration: 01m 13s) [production]
10:37 <_joe_> benchmarking EtcdConfig failure scenarios on mwdebug1001, T185078 [production]
10:22 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1104 - T186321 (duration: 01m 14s) [production]
10:18 <ema> upgrade cache_text @ ulsfo to varnish 5 [production]
10:13 <jynus@tin> Synchronized wmf-config/db-codfw.php: Depool db2073 for maintenance (duration: 01m 12s) [production]
10:08 <moritzm> uploaded Linux 4.9.82-1~wmf1 for jessie-wikimedia to apt.wikimedia.org (retpoline-enabled kernel) [production]
10:00 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1089 with low traffic and depool db1067 - T162807 (duration: 01m 12s) [production]
09:59 <akosiaris> reboot kraz.wikimedia.org (irc.wikimedia.org) [production]
09:47 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1104 - T186321 (duration: 01m 12s) [production]
09:31 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Slowly repool db1104 - T186321 (duration: 01m 12s) [production]
09:20 <marostegui> Stop MySQL on db1104 to switch its binlog to statement - T186321 [production]
09:20 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1104 - T186321 (duration: 01m 13s) [production]
09:19 <moritzm> rebooting multatuli [production]
09:03 <ema> eqiad LVSs: upgrade pybal to 1.14.4 [production]
08:48 <jynus> tendril and dbtree database currently under maintanance [production]
08:47 <ema> codfw LVSs: upgrade pybal to 1.14.4 [production]
08:35 <marostegui> Stop tendril database (db1011) to copy it to db1115 - tendril will be offline while the copy is in progress - T184704 [production]