5601-5650 of 10000 results (40ms)
2015-02-07 §
02:33 <l10nupdate> Synchronized php-1.25wmf16/cache/l10n: (no message) (duration: 00m 01s) [production]
02:20 <LocalisationUpdate> completed (1.25wmf15) at 2015-02-07 02:19:09+00:00 [production]
02:19 <l10nupdate> Synchronized php-1.25wmf15/cache/l10n: (no message) (duration: 00m 02s) [production]
02:11 <qchris> Ran kafka leader re-election as analytics1021 dropped out of it's partition leader role. [production]
01:48 <bblack> leaving cp1064 (jessie upload eqiad) pooled front+back. it's experimental but looks stable. if upload-related 503 spikes and I'm not around, feel free to depool it. [production]
00:18 <qchris> Manually bumping heap for the Hadoop namenodes and revived them after both of them running out of heap and not coming back. [production]
2015-02-06 §
22:53 <marktraceur> Synchronized wmf-config/: [friday] beta config change for tgr (duration: 00m 09s) [production]
22:53 <subbu> restarted parsoid service to kill several stuck processes on multiple nodes [production]
20:05 <robh> ms1004 coming offline, shouldnt page (but disregard if it does) [production]
19:19 <subbu> deployed parsoid hotfiix a9dbd4fc (cherry-pick of 76d6658c) [production]
16:54 <reedy> Synchronized wmf-config/InitialiseSettings.php: Adding cdm16062.contentdm.oclc.org to wgCopyUploadsDomains (duration: 00m 05s) [production]
16:40 <godog> cancel downtime on graphite1001, enable downtime on tungsten pending full decomission [production]
16:05 <godog> bounce ocg on ocg1001 and stop additional ocg instance running [production]
15:50 <bblack> depool -> repool cp1064 varnish-frontend, reduced cache size to 16G, re-enabled compact_memory [production]
15:50 <godog> restart ocg on ocg1003 to pick up statsd dns changes [production]
15:48 <godog> restart ocg on ocg1002 to pick up statsd dns changes [production]
14:33 <bblack> starting up a fresh round of SSL testing on eqiad upload pooling (cp1064) [production]
14:12 <godog> bounce diamond on lvs2004/lvs2005 [production]
13:55 <cmjohnson1> upgrading boron to trusty [production]
10:51 <godog> reimage ms-be2014 [production]
07:50 <_joe_> restarting the parsoid cluster, one node at a time, some processes are stuck. [production]
02:04 <LocalisationUpdate> failed: git pull of extensions failed [production]
01:01 <ori> restarting xenon on fluorine [production]
00:14 <krenair> Synchronized php-1.25wmf15/includes/CategoryViewer.php: https://gerrit.wikimedia.org/r/#/c/188944/1 (duration: 00m 06s) [production]
00:13 <krenair> Synchronized php-1.25wmf16/includes/CategoryViewer.php: https://gerrit.wikimedia.org/r/#/c/188945/1 (duration: 00m 06s) [production]
2015-02-05 §
23:53 <bd808> Updated Wikimania Scholarships to 0852585 (re-enable language selection) + local hack in trebuchet repo to remove incomplete translations [production]
22:53 <reedy> Synchronized wmf-config/InitialiseSettings.php: Enable Parsoid on wikitech (duration: 00m 05s) [production]
21:08 <reedy> Purged l10n cache for 1.25wmf13 [production]
21:03 <reedy> Synchronized php-1.25wmf16/extensions/CheckUser/: (no message) (duration: 00m 07s) [production]
21:00 <reedy> Synchronized php-1.25wmf16: (no message) (duration: 00m 52s) [production]
20:59 <bblack> cp1064 upload b ackend re-enabled in cache.pp; if upload-related 503s ensue later today and I'm not around, feel free to re-disable it [production]
20:57 <mutante> radon - reinstalling, scheduled downtime [production]
20:42 <Reedy> mw1092 giving file has vanished: "/wmf-config/.InitialiseSettings.php.KSg3AF" (in common) [production]
20:42 <reedy> Synchronized wmf-config/: GlobalUserPage and I33a855cecfbe25003fe9e4f5e2fab2f928c79da4 (duration: 00m 07s) [production]
20:41 <reedy> Synchronized wmf-config/: GlobalUserPage and I33a855cecfbe25003fe9e4f5e2fab2f928c79da4 (duration: 00m 05s) [production]
20:35 <reedy> Synchronized wmf-config/: GlobalUserPage and I33a855cecfbe25003fe9e4f5e2fab2f928c79da4 (duration: 00m 08s) [production]
20:34 <reedy> Synchronized php-1.25wmf16/includes/EditPage.php: Id376f9e75c43c5bd0fa910b04d066e6aa37c73d1 (duration: 00m 07s) [production]
20:30 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.25wmf16 [production]
20:29 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedias to 1.25wmf15 [production]
20:21 <reedy> Finished scap: testwiki to 1.25wmf16 (duration: 38m 37s) [production]
20:08 <_joe_> re-exported nfs exports on dataset1001, remounted /mnt/data on snapshot1001 [production]
19:42 <reedy> Started scap: testwiki to 1.25wmf16 [production]
19:41 <reedy> scap aborted: testwiki to 1.25wmf16 (duration: 03m 27s) [production]
19:37 <reedy> Started scap: testwiki to 1.25wmf16 [production]
19:10 <legoktm> clearing bad sidebar memcache entries on commonswiki [production]
18:08 <_joe_> restarting nutcracker on jobrunners [production]
18:06 <_joe_> restarting nutcracker on api appservers [production]
18:01 <ori> restarting nutcracker on all appservers [production]
17:54 <ori> Synchronized wmf-config/InitialiseSettings.php: I4f28205e6: Set $wmgUseMonologLogger to false (duration: 00m 06s) [production]
17:47 <ori> Synchronized wmf-config/logging.php: Live hack: disable Logstash logging on suspicion that it is acting up (duration: 00m 05s) [production]