3551-3600 of 10000 results (50ms)
2019-03-26 §
09:09 <filippo@puppetmaster1001> conftool action : set/pooled=no; selector: name=prometheus1004.eqiad.wmnet [production]
09:05 <filippo@puppetmaster1001> conftool action : set/pooled=yes; selector: name=prometheus1003.eqiad.wmnet [production]
08:09 <marostegui> Deploy schema change on s2 codfw master, this will generate lag on codfw s2 [production]
07:16 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1119 (duration: 00m 49s) [production]
06:51 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1119 (duration: 00m 50s) [production]
06:35 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1106 (duration: 00m 52s) [production]
06:02 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1106 (duration: 00m 51s) [production]
06:02 <marostegui> Deploy schema change on db1106, this will generate lag on s1 on labs hosts [production]
2019-03-25 §
23:20 <jforrester@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT T219234 Turn on Elastica logging channel (duration: 00m 51s) [production]
22:32 <krinkle@deploy1001> Synchronized docroot/wikipedia.org/speed-tests/Banksy.enwiki.872156204: T185446 (duration: 00m 49s) [production]
21:44 <jforrester@deploy1001> Synchronized wmf-config/Wikibase.php: T216206 Deploy WikibaseLexemeCirrusSearch: Part 1 - set up variables, sub-part b (duration: 00m 49s) [production]
21:43 <jforrester@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T216206 Deploy WikibaseLexemeCirrusSearch: Part 1 - set up variables, sub-part a (duration: 00m 50s) [production]
21:40 <XioNoX> apply transport-in4 filter to cr1/2-eqiad - T190090 [production]
21:33 <jforrester@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T218715 Enable WBCS on Testcommons too (duration: 00m 50s) [production]
20:32 <ebernhardson> T218994 set various deprecation channels on all six cirrus elasticsearch clusters to ERROR [production]
19:54 <dcausse> elasticsearch search cluster: SET "logger.org.elasticsearch.common.logging.DeprecationLogger" to "ERROR" to psi/omega@eqiad (T218994) [production]
19:48 <dcausse> elasticsearch search cluster: SET "logger.org.elasticsearch.deprecation.index.query.functionscore.ScoreFunctionBuilder" to "ERROR" to chi/psi/omega@eqiad (T218994) [production]
19:40 <volans> restart icinga on icinga1001 to reset modified attributes [production]
19:37 <dcausse> morning SWAT done [production]
19:33 <dcausse@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [cirrus] switch all wikis to eqiad (elastic 6.5.4) (duration: 00m 50s) [production]
19:21 <dcausse@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T192254 (duration: 00m 49s) [production]
19:13 <dcausse@deploy1001> Synchronized wmf-config/CommonSettings.php: T218260 (duration: 00m 49s) [production]
19:06 <ebernhardson@deploy1001> Finished deploy [search/mjolnir/deploy@1cbf290]: Roll update to mjolnir-bulk-daemon es6 handling of super_detect_noop (duration: 03m 27s) [production]
19:02 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@1cbf290]: Roll update to mjolnir-bulk-daemon es6 handling of super_detect_noop [production]
18:46 <dcausse@deploy1001> Synchronized wmf-config/flaggedrevs.php: revert T217507 (duration: 00m 49s) [production]
18:43 <ebernhardson> restart mjolnir-kafka-msearch-daemon across cirrus elasticsearch servers [production]
18:41 <dcausse@deploy1001> scap failed: average error rate on 6/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/db09a36be5ed3e81155041f7d46ad040 for details) [production]
18:32 <dcausse@deploy1001> Synchronized php-1.33.0-wmf.22/extensions/FlaggedRevs/: T218949: Fix reject changes when user is partially blocked (duration: 00m 51s) [production]
18:26 <dcausse@deploy1001> Synchronized dblists/visualeditor-nondefault.dblist: T192135 (duration: 00m 50s) [production]
18:15 <dcausse@deploy1001> Synchronized wmf-config/CommonSettings.php: T211622: Enforce 8 char password length requirements for non-privileged users (duration: 00m 50s) [production]
17:24 <onimisionipe@deploy1001> Finished deploy [wdqs/wdqs@281aaf8]: New build for blazegraph and updater plus GUI updates (duration: 10m 31s) [production]
17:24 <elukey> restart pdfrender on scb1004 [production]
17:14 <onimisionipe@deploy1001> Started deploy [wdqs/wdqs@281aaf8]: New build for blazegraph and updater plus GUI updates [production]
17:11 <ebernhardson> restart mjolnir-kafka-msearch-daemon on relforge100[12] [production]
17:10 <dcausse@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T218878: [cirrus] switch low volume wikis to eqiad (elastic 6.5.4) (duration: 00m 49s) [production]
16:56 <ebernhardson@deploy1001> Finished deploy [search/mjolnir/deploy@2fb5038]: Ship new logging support code via new simplified virtualenv deployment (duration: 09m 52s) [production]
16:47 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@2fb5038]: Ship new logging support code via new simplified virtualenv deployment [production]
16:28 <ebernhardson@deploy1001> Finished deploy [search/mjolnir/deploy@ba09eb5]: Ship new logging support code via new simplified virtualenv deployment (duration: 09m 10s) [production]
16:19 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@ba09eb5]: Ship new logging support code via new simplified virtualenv deployment [production]
16:19 <hashar> updating Jenkins plugins and restarting [production]
16:16 <ebernhardson@deploy1001> Finished deploy [search/mjolnir/deploy@6eda7d8]: Ship new logging support code via new simplified virtualenv deployment (duration: 02m 38s) [production]
16:13 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@6eda7d8]: Ship new logging support code via new simplified virtualenv deployment [production]
15:48 <XioNoX> remove 2nd AS7568 router in Equinix Singapore [production]
15:21 <ebernhardson@deploy1001> Finished deploy [search/mjolnir/deploy@ddf26d0]: Ship new logging support code (duration: 01m 29s) [production]
15:20 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@ddf26d0]: Ship new logging support code [production]
15:00 <jbond42> updateing passenger on rhodium [production]
14:29 <andrewbogott> updating slapd indexes on seaborgium, serpens, ldap-eqiad-replica01, ldap-eqiad-replica02 for 498396 [production]
13:52 <ema@puppetmaster1001> conftool action : set/pooled=yes; selector: name=cp1076.eqiad.wmnet,service=varnish-fe [production]
13:52 <ema@puppetmaster1001> conftool action : set/pooled=yes; selector: name=cp1076.eqiad.wmnet,service=nginx [production]
13:52 <ema> cp1076: repool varnish-fe, frontend misses served by cp-ats T213263 [production]