4501-4550 of 10000 results (54ms)
2018-03-30 §
11:40 <dcausse> elastic@codfw cluster restarts complete (T189239) [production]
10:55 <dcausse> resuming elastic@codfw cluster restarts [production]
10:17 <elukey> roll restart of zookeeper daemons on druid100[123] (Druid analytics cluster) to pick up the new prometheus jmx agent [production]
09:31 <elukey> restart oozie/hive daemons on an1003 for openjdk-8 upgrades [production]
08:38 <elukey> rolling restart of hadoop-hdfs-datanode on all the hadoop worker nodes after https://gerrit.wikimedia.org/r/423000 [production]
07:39 <elukey> rolling restart of yarn-hadoop-nodemanagers on all the hadoop worker nodes after https://gerrit.wikimedia.org/r/423000 [production]
2018-03-29 §
23:47 <ebernhardson@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: T189252: Enable perf oversampling for remaining countries in Asia (duration: 01m 16s) [production]
23:40 <ebernhardson@tin> Synchronized php-1.31.0-wmf.27/extensions/WikimediaEvents/modules/all/ext.wikimediaEvents.searchSatisfaction.js: SWAT: T187148: Start cirrus AB test (duration: 01m 16s) [production]
23:37 <ebernhardson@tin> Synchronized php-1.31.0-wmf.26/extensions/WikimediaEvents/modules/all/ext.wikimediaEvents.searchSatisfaction.js: SWAT: T187148: Start cirrus AB test (duration: 01m 16s) [production]
23:12 <ebernhardson@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: T187148: Configure 5 buckets for cirrus AB test (duration: 01m 17s) [production]
22:10 <andrew@tin> Finished deploy [horizon/deploy@14d3e7d]: Updating Horizon with possible fix for T189706 (duration: 03m 16s) [production]
22:06 <andrew@tin> Started deploy [horizon/deploy@14d3e7d]: Updating Horizon with possible fix for T189706 [production]
20:07 <robh> shuttdown cp2022 for hw testing [production]
18:49 <maxsem@tin> Synchronized php-1.31.0-wmf.27/skins/MinervaNeue: https://gerrit.wikimedia.org/r/#/c/423012/ (duration: 01m 17s) [production]
18:27 <maxsem@tin> Synchronized php-1.31.0-wmf.26/includes/: Shorten summary length to 500 (duration: 02m 06s) [production]
18:22 <maxsem@tin> Synchronized php-1.31.0-wmf.27/includes/: Shorten summary length to 500 (duration: 02m 14s) [production]
17:55 <dcausse> pausing restarts of elastic@codfw (6 nodes left) [production]
17:35 <mobrovac@tin> Finished deploy [restbase/deploy@af592d6]: Add bawikibooks - T191033 (duration: 30m 35s) [production]
17:30 <demon@tin> Synchronized docroot/wwwportal/w/search-redirect.php: removing symlink indirection (duration: 01m 16s) [production]
17:05 <mobrovac@tin> Started deploy [restbase/deploy@af592d6]: Add bawikibooks - T191033 [production]
14:54 <mobrovac@tin> Synchronized wmf-config/jobqueue.php: Cleanup: Use only EventBus for refreshLinks - T185052 (duration: 01m 18s) [production]
14:00 <moritzm> restarting parsoid and related service on ruthenium to pick up openssl update [production]
13:52 <dcausse> reverted and rebased tin for undeployed patch due to scap issues (https://gerrit.wikimedia.org/r/#/c/422906/ https://gerrit.wikimedia.org/r/#/c/422929/) [production]
13:34 <dcausse> aborted scap sync-dir php-1.31.0-wmf.27/extensions/CirrusSearch/ (was taking too much time at: waiting on sync-masters, ok: 1, left: 1) [production]
12:54 <moritzm> installing ICU security updates on trusty [production]
12:29 <dcausse> recreating replicas for skwiki_content in elastic@codfw due to stalled shard recovery [production]
12:18 <ariel@tin> Finished deploy [dumps/dumps@982cebd]: ability to configure production of recombined metacurrent page content file (duration: 00m 02s) [production]
12:18 <ariel@tin> Started deploy [dumps/dumps@982cebd]: ability to configure production of recombined metacurrent page content file [production]
11:02 <ariel@tin> Finished deploy [dumps/dumps@96ba844]: cleanup 'latest' links, rss files from old runs (duration: 00m 04s) [production]
11:02 <ariel@tin> Started deploy [dumps/dumps@96ba844]: cleanup 'latest' links, rss files from old runs [production]
10:50 <dcausse> restarting elastic@codfw for JVM and plugin upgrade (T189239) [production]
09:16 <elukey> roll restart aqs on aqs100* for icu/openssl upgrades [production]
08:18 <akosiaris> T189075 upload apertium_3.5.1-1+wmf1 to apt.wikimedia.org/jessie-wikimedia/main [production]
08:18 <moritzm> installing OpenJDK security updates on elastic* hosts (along with current version of the search plugins package) [production]
08:07 <elukey> roll restart of cassandra on aqs* for openjdk-8 upgrades [production]
07:20 <moritzm> installing openssl security updates [production]
07:18 <ema> reboot cache@eqiad for retpoline kernel updates: T188092 [production]
04:35 <twentyafterfour> ran scap pull on deploy1001 [production]
02:00 <l10nupdate@tin> LocalisationUpdate failed: git pull of core failed [production]
2018-03-28 §
23:50 <eileen> update civicrm revision changed from 9478ca39f1 to d6855cd281 (further security module updates, engage import dedupe) [production]
23:38 <ebernhardson@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: T187148: Configure next Cirrus AB test (duration: 01m 16s) [production]
23:18 <ebernhardson@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: T184969: Enable PageAssessments on trwiki (duration: 01m 09s) [production]
23:13 <MaxSem> created PageAssessments tables on trwiki [production]
22:17 <twentyafterfour@tin> Synchronized php: group1 wikis to 1.31.0-wmf.26 (duration: 01m 18s) [production]
22:16 <twentyafterfour@tin> rebuilt and synchronized wikiversions files: group1 wikis to 1.31.0-wmf.26 [production]
22:13 <twentyafterfour> deploy of 1.31.0-wmf.27 resulted in a lot of SlowTimer errors for SlowTimer [10000ms] at runtime/ext_mysql: slow query: SELECT MASTER_GTID_WAIT(...) [production]
22:12 <eileen> civicrm revision changed from 3f6028b24f to 9478ca39f1 (drupal security update) [production]
22:09 <twentyafterfour@tin> rebuilt and synchronized wikiversions files: sync https://gerrit.wikimedia.org/r/#/c/422563/ group1 wikis to 1.31.0-wmf.27 refs T183966 T190960 [production]
22:08 <twentyafterfour> rolling forward group1 to 1.31.0-wmf.27 refs T183966 T190960 [production]
22:05 <twentyafterfour@tin> Synchronized php-1.31.0-wmf.27/includes/: sync https://gerrit.wikimedia.org/r/#/c/422565/ (duration: 02m 15s) [production]