|
2018-03-30
§
|
| 12:47 |
<akosiaris> |
T189076 upload apertium-fra to apt.wikimedia.org/jessie-wikimedia/main |
[production] |
| 11:44 |
<dcausse> |
running forceSearchIndex from terbium to cleanup elastic indices for (testwiki, mediawikiwiki, labswiki, labtestwiki, svwiki) (T189694) |
[production] |
| 11:40 |
<dcausse> |
elastic@codfw cluster restarts complete (T189239) |
[production] |
| 11:38 |
<dcausse> |
deployment-prep reindexing with forceSearchIndex all beta wikis (T189694) |
[releng] |
| 10:55 |
<dcausse> |
resuming elastic@codfw cluster restarts |
[production] |
| 10:17 |
<elukey> |
roll restart of zookeeper daemons on druid100[123] (Druid analytics cluster) to pick up the new prometheus jmx agent |
[production] |
| 09:56 |
<hashar> |
Nuking /srv/zuul/git/labs/tools/stewardbots on zuul-merger hosts (contint1001 and contint2001). Fetch fails with org.eclipse.jgit.transport.UploadPackInternalServerErrorException | T191077 |
[releng] |
| 09:56 |
<hashar> |
Nuking /srv/zuul/git/labs/tools/stewardbots on zuul-merger hosts (contint1001 and contint2001). Fetch fails with org.eclipse.jgit.transport.UploadPackInternalServerErrorException | TT191077 |
[releng] |
| 09:44 |
<elukey> |
re-enable camus |
[analytics] |
| 09:31 |
<elukey> |
restart oozie/hive daemons on an1003 for openjdk-8 upgrades |
[production] |
| 08:38 |
<elukey> |
rolling restart of hadoop-hdfs-datanode on all the hadoop worker nodes after https://gerrit.wikimedia.org/r/423000 |
[production] |
| 08:26 |
<elukey> |
stopped camus to drain the cluster - prep for easy restart of analytics1003's jvm daemons |
[analytics] |
| 07:39 |
<elukey> |
rolling restart of yarn-hadoop-nodemanagers on all the hadoop worker nodes after https://gerrit.wikimedia.org/r/423000 |
[production] |
|
2018-03-29
§
|
| 23:47 |
<ebernhardson@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: T189252: Enable perf oversampling for remaining countries in Asia (duration: 01m 16s) |
[production] |
| 23:40 |
<ebernhardson@tin> |
Synchronized php-1.31.0-wmf.27/extensions/WikimediaEvents/modules/all/ext.wikimediaEvents.searchSatisfaction.js: SWAT: T187148: Start cirrus AB test (duration: 01m 16s) |
[production] |
| 23:37 |
<ebernhardson@tin> |
Synchronized php-1.31.0-wmf.26/extensions/WikimediaEvents/modules/all/ext.wikimediaEvents.searchSatisfaction.js: SWAT: T187148: Start cirrus AB test (duration: 01m 16s) |
[production] |
| 23:12 |
<ebernhardson@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: T187148: Configure 5 buckets for cirrus AB test (duration: 01m 17s) |
[production] |
| 22:10 |
<andrew@tin> |
Finished deploy [horizon/deploy@14d3e7d]: Updating Horizon with possible fix for T189706 (duration: 03m 16s) |
[production] |
| 22:06 |
<andrew@tin> |
Started deploy [horizon/deploy@14d3e7d]: Updating Horizon with possible fix for T189706 |
[production] |
| 20:55 |
<milimetric> |
accidentally killed mediawiki-geowiki-monthly-coord, and then restarted it |
[analytics] |
| 20:12 |
<ottomata> |
blacklisted mediawiki.job topics from main -> jumbo MirrorMaker again, don't want to page over the weekend while this still is not stable. T189464 |
[analytics] |
| 20:09 |
<chicocvenancio> |
killed interactive processes in tools-bastion-03 |
[tools] |
| 20:07 |
<robh> |
shuttdown cp2022 for hw testing |
[production] |
| 19:56 |
<chicocvenancio> |
several interactive jobs running in bastion-03. I am writing to connected users and will kill the jobs once done |
[tools] |
| 19:39 |
<Amir1> |
ladsgroup@deployment-tin:~$ mwscript updateCollation.php --wiki=fawiki (T190965) |
[releng] |
| 19:33 |
<hashar> |
docker build image releng/quibble-stretch:0.0.2 https://gerrit.wikimedia.org/r/#/c/423025/ |
[releng] |
| 18:49 |
<maxsem@tin> |
Synchronized php-1.31.0-wmf.27/skins/MinervaNeue: https://gerrit.wikimedia.org/r/#/c/423012/ (duration: 01m 17s) |
[production] |
| 18:27 |
<maxsem@tin> |
Synchronized php-1.31.0-wmf.26/includes/: Shorten summary length to 500 (duration: 02m 06s) |
[production] |
| 18:22 |
<maxsem@tin> |
Synchronized php-1.31.0-wmf.27/includes/: Shorten summary length to 500 (duration: 02m 14s) |
[production] |
| 18:08 |
<greg-g> |
ran scap task-series blockers --count 26 --date 2018-04-23 1.32.0-wmf.1 |
[releng] |
| 17:55 |
<dcausse> |
pausing restarts of elastic@codfw (6 nodes left) |
[production] |
| 17:35 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@af592d6]: Add bawikibooks - T191033 (duration: 30m 35s) |
[production] |
| 17:30 |
<demon@tin> |
Synchronized docroot/wwwportal/w/search-redirect.php: removing symlink indirection (duration: 01m 16s) |
[production] |
| 17:18 |
<zhuyifei1999_> |
upgraded youtube_dl in frontend from 2017.9.2 to 2018.3.26.1 |
[video] |
| 17:05 |
<mobrovac@tin> |
Started deploy [restbase/deploy@af592d6]: Add bawikibooks - T191033 |
[production] |
| 16:44 |
<hashar> |
rebuilding docker images for releng/quibble:0.0.1 https://gerrit.wikimedia.org/r/#/c/422959/ |
[releng] |
| 16:38 |
<hashar> |
Reloading Zuul for https://gerrit.wikimedia.org/r/#/c/422482/ |
[releng] |
| 16:35 |
<greg-g> |
brought compiler02.puppet3-diffs.eqiad.wmflabs back online (herron asked) |
[releng] |
| 15:13 |
<paladox> |
creating new repo mediawiki/extensions/SecureAuth per https://www.mediawiki.org/wiki/Gerrit/New_repositories/Requests |
[releng] |
| 14:55 |
<mutante> |
removing deploy1001 from scap masters, keeping it as scap host. to avoid issues syncing (slower) to 3 masters but keep it synced as a regular host (T191029) |
[releng] |
| 14:54 |
<mobrovac@tin> |
Synchronized wmf-config/jobqueue.php: Cleanup: Use only EventBus for refreshLinks - T185052 (duration: 01m 18s) |
[production] |
| 14:26 |
<elukey> |
removed from deployment-puppetmaster02's ops repo "1ffd6e72a9 WIP Bug T184234" due to git sync failures |
[releng] |
| 14:00 |
<moritzm> |
restarting parsoid and related service on ruthenium to pick up openssl update |
[production] |
| 13:52 |
<dcausse> |
reverted and rebased tin for undeployed patch due to scap issues (https://gerrit.wikimedia.org/r/#/c/422906/ https://gerrit.wikimedia.org/r/#/c/422929/) |
[production] |
| 13:34 |
<dcausse> |
aborted scap sync-dir php-1.31.0-wmf.27/extensions/CirrusSearch/ (was taking too much time at: waiting on sync-masters, ok: 1, left: 1) |
[production] |
| 12:54 |
<moritzm> |
installing ICU security updates on trusty |
[production] |
| 12:29 |
<dcausse> |
recreating replicas for skwiki_content in elastic@codfw due to stalled shard recovery |
[production] |
| 12:18 |
<ariel@tin> |
Finished deploy [dumps/dumps@982cebd]: ability to configure production of recombined metacurrent page content file (duration: 00m 02s) |
[production] |
| 12:18 |
<ariel@tin> |
Started deploy [dumps/dumps@982cebd]: ability to configure production of recombined metacurrent page content file |
[production] |
| 11:02 |
<ariel@tin> |
Finished deploy [dumps/dumps@96ba844]: cleanup 'latest' links, rss files from old runs (duration: 00m 04s) |
[production] |