1151-1200 of 10000 results (31ms)
2016-07-13 ยง
21:55 <mutante> ytterbium - puppet enabled again, fix deployed [production]
21:48 <mutante> ytterbium, disabled puppet, started apache, needs fix [production]
20:40 <demon@tin> Synchronized README: no-op to bring co-masters in sync (duration: 00m 28s) [production]
20:27 <bearND> deployed mobileapps d1eb1da [production]
20:20 <demon@tin> rebuilt wikiversions.php and synchronized wikiversions files: group1 to wmf.10 [production]
20:19 <bearND> starting mobileapps deploy [production]
20:03 <ebernhard|lunch> Update codfw elasticsearch cluster sttings with cluster.routing.allocation.disk.watermark.low: 70% to match eqiad and reduce free space icinga warnings [production]
19:28 <ostriches> gerrit: restarting, puppet back on, issue fixed. [production]
19:23 <anomie@tin> Synchronized php-1.28.0-wmf.10/includes/auth/AuthManager.php: Add timing data logging for T119736 (duration: 00m 28s) [production]
19:23 <anomie@tin> Synchronized php-1.28.0-wmf.8/includes/auth/AuthManager.php: Add timing data logging for T119736 (duration: 00m 27s) [production]
19:14 <legoktm@tin> Synchronized php-1.28.0-wmf.8/includes/api/ApiQueryRecentChanges.php: API: Remove index forcing in ApiQueryRecentChanges - T140108 (duration: 00m 26s) [production]
19:04 <demon@tin> rebuilt wikiversions.php and synchronized wikiversions files: all group0 to wmf.10 [production]
18:41 <ostriches> gerrit/ytterbium: flapped for a minute because of incompat 2.12/2.8 config. Working, puppet disabled pending real fix. [production]
18:40 <mutante> gerrit has a temp problem. maintenance going on [production]
18:34 <krenair@tin> Synchronized wmf-config: labs-only change, should be a noop here: https://gerrit.wikimedia.org/r/298812 (duration: 00m 27s) [production]
18:32 <mutante> gerrit will restart shortly for a config change. expect a very short downtime [production]
18:22 <legoktm> checkLocalUser.php finished, starting run #2 now [production]
17:58 <legoktm@tin> Synchronized wmf-config/: PoolCounterClient.php -> extension.json (duration: 00m 32s) [production]
17:44 <legoktm@tin> Synchronized php-1.28.0-wmf.10/extensions/WikimediaEvents/WikimediaEventsHooks.php: Include the namespace for all pages & Include the resolved special page name for special pages - T138500 (duration: 00m 36s) [production]
17:41 <demon@tin> Finished scap: wmf.10 code sync + testwiki to wmf.10 for l10n cache gen (once more with feeling) (duration: 47m 06s) [production]
17:35 <ejegg> turned on cURL verbose logging for AstroPay requests on payments [production]
17:27 <jynus> drop databases fab_migration, percona and test from m3 T138460 [production]
17:25 <_joe_> restarting hhvm on mw1229 (stuck in HPHP::Treadmill::getAgeOldestRequest) [production]
17:22 <urandom> Starting restbase on restbase1013.eqiad.wmnet [production]
16:54 <demon@tin> Started scap: wmf.10 code sync + testwiki to wmf.10 for l10n cache gen (once more with feeling) [production]
16:50 <ejegg> updated CentralNotice for cookie cleanup [production]
16:48 <ejegg@tin> Synchronized php-1.28.0-wmf.8/extensions/CentralNotice/: (no message) (duration: 01m 52s) [production]
16:47 <demon@tin> scap failed: OSError [Errno 1] Operation not permitted: '/var/lock/scap' (duration: 00m 00s) [production]
16:41 <demon@tin> scap aborted: wmf.10 code sync + testwiki to wmf.10 for l10n cache gen (duration: 00m 37s) [production]
16:40 <demon@tin> Started scap: wmf.10 code sync + testwiki to wmf.10 for l10n cache gen [production]
16:39 <demon@tin> Purged l10n cache for 1.28.0-wmf.9 [production]
16:38 <demon@tin> Purged l10n cache for 1.28.0-wmf.7 [production]
16:38 <demon@tin> Purged l10n cache for 1.28.0-wmf.6 [production]
16:37 <demon@tin> Purged l10n cache for 1.28.0-wmf.5 [production]
16:19 <hashar> CI slightly overloaded / backloaded due to a long tail of Wikibase changes sent in Gerrit. [production]
16:03 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings-labs.php: SWAT: [[gerrit:298786|[Beta] Change ORES thresholds in beta]] (duration: 00m 29s) [production]
16:01 <thcipriani@tin> Synchronized wmf-config/LabsServices.php: SWAT: [[gerrit:298782|[Beta] Parsoid: direct traffic to deployment-parsoid07]] (duration: 00m 26s) [production]
15:30 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:298715|Enable ORES review tool for Turkish Wikipedia (T139992)]] (duration: 00m 28s) [production]
15:11 <urandom> Stopping Staging dumps : T139639 [production]
14:44 <urandom> Starting offset dump runs from {xenon,cerium,praseodymium}.eqiad.wmnet : T139639 [production]
14:38 <urandom> Restarting Cassandra on xenon.eqiad.wmnet : T139639 [production]
14:32 <urandom> Restarting RESTBase on xenon.eqiad.wmnet : T139639 [production]
14:29 <yurik> deployed kartotherian https://gerrit.wikimedia.org/r/#/c/298731/ & tilerator https://gerrit.wikimedia.org/r/#/c/298732/ [production]
14:25 <moritzm> depooling mw1298 (image scaler) for some tests [production]
14:21 <godog> reboot ms-be1012, many mkfs.xfs stuck on broken sdh [production]
14:17 <oblivian@palladium> conftool action : delete; selector: cluster=rcstream [production]
14:16 <gehel> shutting down elastic1001-1016 (T139758) [production]
14:16 <oblivian@palladium> conftool action : delete; selector: cluster=rcstream [production]
14:13 <yurik> about to deploy updated kartotherian & tilerator for node 4.4.6 [production]
14:09 <urandom> Dropping legacy system_auth tables in staging to complete RBAC conversion : T139639 [production]