851-900 of 10000 results (50ms)
2017-12-05 ยง
21:12 <demon@tin> Finished scap: bootstrap wmf.11 (duration: 51m 01s) [production]
20:50 <twentyafterfour> phabricator: running `sudo bin/garbage collect --collector search.ferret.ngram` [production]
20:29 <twentyafterfour> phabricator: running `sudo bin/search ngrams --threshold 0.2` [production]
20:21 <demon@tin> Started scap: bootstrap wmf.11 [production]
19:50 <mutante> forcing puppet ron on ores eqiad to reduce number of celery workers used for stress test [production]
19:37 <ppchelko@tin> Finished deploy [eventlogging/eventbus@6ca0372]: Make the kafka async deliver callback thread-safe. T180017 (duration: 01m 29s) [production]
19:36 <ppchelko@tin> Started deploy [eventlogging/eventbus@6ca0372]: Make the kafka async deliver callback thread-safe. T180017 [production]
19:20 <gehel> moving eventlogging collection by logstash from logstash1003 to logstash1007, no messages **should** be lost - T175830 [production]
18:24 <ppchelko@tin> Finished deploy [eventlogging/eventbus@6ca0372]: Make the kafka async deliver callback thread-safe. Limited to kafka1001. T180017 (duration: 00m 14s) [production]
18:24 <ppchelko@tin> Started deploy [eventlogging/eventbus@6ca0372]: Make the kafka async deliver callback thread-safe. Limited to kafka1001. T180017 [production]
17:27 <jmm@puppetmaster1001> conftool action : set/pooled=yes; selector: mw2118.codfw.wmnet [production]
15:56 <awight> beginning stress test on ores* (non-production) [production]
15:54 <awight@tin> Finished deploy [ores/deploy@6baed71]: (non-production) Test ORES deployment to ores100[1-2] (duration: 01m 01s) [production]
15:53 <awight@tin> Started deploy [ores/deploy@6baed71]: (non-production) Test ORES deployment to ores100[1-2] [production]
15:15 <ottomata> restarrting kafka-jumbo brokers, applying SSL (downtime scheduled) [production]
15:03 <urandom> bootstrapping cassandra, restbase1014-c - T179422 [production]
14:54 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Fully repool db1076 (duration: 00m 43s) [production]
14:41 <zeljkof> EU SWAT finished [production]
14:40 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:395540|Revert "Add category collation for sewiki" (T181503)]] (duration: 00m 44s) [production]
14:15 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:395009|Enable RemexHTML on itwiki and dewiki (T181188 T181190)]] (duration: 00m 43s) [production]
13:39 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1076 weight (duration: 00m 44s) [production]
13:14 <moritzm> reimaging mw2118/mw2119 (video scalers) to stretch [production]
12:21 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Fully repool db1034 (duration: 00m 43s) [production]
12:13 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1076 with low weight (duration: 00m 44s) [production]
11:58 <marostegui> Upgrade MariaDB and kernel on db1076 [production]
11:53 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1034 - T178359! (duration: 00m 43s) [production]
10:53 <moritzm> rebooting meitnerium/archiva.wikimedia.org for update to 4.9.51 [production]
10:46 <moritzm> rebooting einsteinium (icinga.wikimedia.org) for update to 4.9,51 [production]
10:45 <elukey> reboot druid1003 for kernel+jvm updates - T179943 [production]
10:33 <addshore@tin> Synchronized wmf-config/extension-list: [[gerrit:394283|extension-list extension.json entrypoint for ArticlePlaceholder]] (duration: 00m 43s) [production]
10:33 <moritzm> rebooting tegmen for update to 4.9,51 [production]
10:07 <jynus> stopping dbstore1002 (s5) and dbstore2001 (s5) for maintenance [production]
10:05 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1034 - T178359! (duration: 00m 43s) [production]
10:00 <ema> cp4021: restart varnish-be due to mbox lag/fetch failures [production]
09:49 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1034 with low weight - T178359! (duration: 00m 43s) [production]
09:42 <elukey> reboot analytics100[12] for kernel+jvm updates (Hadoop Master nodes) - T179943 [production]
09:39 <godog> bootstrap restbase1014-b - T179422 [production]
09:20 <marostegui> Optimize s7 on db1098 - T178359 [production]
09:03 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1076 - T174569 (duration: 00m 43s) [production]
09:01 <marostegui> Deploy schema change on db1076 (s2) - T174569 [production]
08:57 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1090 - T174569 (duration: 00m 44s) [production]
08:54 <moritzm> enabling test production traffic for mw1259 (stretch-based video scaler) [production]
08:24 <jmm@puppetmaster1001> conftool action : set/pooled=yes; selector: mw1259.eqiad.wmnet [production]
08:08 <moritzm> installing python updates on trusty [production]
07:27 <kartik@tin> Finished deploy [cxserver/deploy@4b74f03]: Update cxserver to 1693bcf (duration: 03m 22s) [production]
07:23 <kartik@tin> Started deploy [cxserver/deploy@4b74f03]: Update cxserver to 1693bcf [production]
06:58 <marostegui> Stop MySQL on db1034 to clone db1098:3317 - T178359 [production]
06:55 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1034 - T178359! (duration: 00m 43s) [production]
06:48 <marostegui> Fix dbstore1002 replication [production]
06:20 <marostegui> Deploy schema change on db1090 (s2) - T174569 [production]