201-250 of 10000 results (26ms)
2016-04-07 §
23:13 <dereckson@tin> Synchronized wmf-config/InitialiseSettings.php: Raise upload-by-URL request timeout (T118887) (duration: 00m 41s) [production]
22:05 <Pchelolo> finished update RESTBase to 7f69f86ee9 [production]
22:00 <Pchelolo> start update RESTBase to 7f69f86ee9 [production]
21:32 <mattflaschen@tin> Synchronized wmf-config/LabsServices.php: Beta Cluster change (duration: 00m 30s) [production]
21:30 <krinkle@tin> Synchronized w/: I69653efe0f1968: rm old symlinks (duration: 00m 46s) [production]
19:55 <bblack> stop->disable varnishrls service on non-text clusters (upload, maps, misc) - ( https://gerrit.wikimedia.org/r/#/c/281439/ ) [production]
19:34 <ottomata> restarting eventlogging so it runs out of the scap deploy in eventlogging/analytics [production]
19:15 <dduvall@tin> rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.27.0-wmf.20 [production]
19:00 <ejegg> updated paymentswiki fraud filter settings [production]
16:48 <godog> start raid expansion on restbase2002 T127951 [production]
16:37 <godog> repool restbase2003, raid expansion finished, depool restbase2002 [production]
15:30 <thcipriani@tin> Synchronized wmf-config/ProductionServices.php: SWAT: Use local resources in codfw for parsoid, url-downloader and mathoid [[gerrit:279355]] (duration: 00m 25s) [production]
15:22 <thcipriani@tin> Synchronized wmf-config: SWAT: Use ProductionServices for the jobqueue configuration 3/3 [[gerrit:279350]] (duration: 00m 31s) [production]
15:22 <thcipriani@tin> Synchronized wmf-config/jobqueue.php: SWAT: Use ProductionServices for the jobqueue configuration 2/3 [[gerrit:279350]] (duration: 00m 27s) [production]
15:21 <thcipriani@tin> Synchronized wmf-config/ProductionServices.php: SWAT: Use ProductionServices for the jobqueue configuration 1/3 [[gerrit:279350]] (duration: 00m 30s) [production]
15:06 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable VisualEditor Beta Feature on Wikisources, Wiktionaries [[gerrit:280828]] (duration: 00m 30s) [production]
14:47 <gehel@tin> Synchronized wmf-config: switching CirrusSearch to use Elasticsearch cluster in codfw (duration: 00m 30s) [production]
14:47 <gehel> syncing config to activate switch of CirrusSearch to codfw [production]
14:35 <gehel> switching CirrusSearch to use Elasticsearch cluster in codfw (again), testing on mw1017 first [production]
13:58 <chasemp> labstore restart nfs-export that crashed [production]
13:53 <gehel@tin> Synchronized wmf-config: switching CirrusSearch to use Elasticsearch cluster in codfw (duration: 00m 35s) [production]
13:50 <gehel@tin> Synchronized wmf-config: switching CirrusSearch to use Elasticsearch cluster in codfw (duration: 00m 31s) [production]
13:48 <gehel> switching CirrusSearch to use Elasticsearch cluster in codfw [production]
13:28 <_joe_> restarted hhvm on mw1015 [production]
13:22 <gehel@tin> Synchronized wmf-config: Fix TTMServer elastic config (duration: 00m 32s) [production]
12:50 <hoo@tin> Synchronized wmf-config/Wikibase.php: Bump $wgCacheEpoch on Wikidata after Property conversions (duration: 00m 26s) [production]
12:49 <_joe_> restarting hhvm on mw1213, deadlock in HPHP::Treadmill::getAgeOldestRequest [production]
12:46 <gehel@tin> Synchronized wmf-config: Point the codfw label back to the codfw cluster (duration: 00m 27s) [production]
12:37 <gehel@tin> Synchronized wmf-config: Don't use HTTPS+pooling for labswiki (duration: 00m 27s) [production]
12:22 <gehel@tin> Synchronized wmf-config: (no message) (duration: 00m 35s) [production]
12:08 <gehel> enabling HTTPS + connection pooling for CirrusSerach on all mediawiki nodes (T131839) [production]
12:06 <godog> remove restbase2004-b cassandra data directory [production]
11:57 <godog> bump raid rebuild to 20MB/s on restbase2003 [production]
10:03 <_joe_> restarted hhvm on mw1114 (one of the child processes was defunct) and on mw1173 (deadlock in HPHP::Treadmill::getAgeOldestRequest) [production]
09:35 <gehel> deploying HHVM HTTP pool sizing on all MW nodes (T131839 / https://gerrit.wikimedia.org/r/#/c/281881), not used yet, no impact expected [production]
09:28 <elukey> de-pooling/re-pooling aqs100[23].eqiad.wmnet for nodejs upgrade [production]
09:19 <elukey> re-added aqs1001.eqiad.wmnet to LVS pool via confctl [production]
08:59 <elukey> removed aqs1001.eqiad.wmnet from LVS pool via confd for nodejs upgrade [production]
08:22 <godog> stop cassandra bootstrap of restbase2004-b, not enough disk [production]
08:17 <akosiaris> renamed duplicate cxserver-deploy and cxserver repos in phabricator to citoid-deploy and citoid respectively [production]
07:04 <volans@tin> Synchronized wmf-config/db-codfw.php: Depool crashed db2047, needs to be reimaged T132011 (duration: 00m 38s) [production]
03:08 <bd808> backfilled missing SAL entries from 2016-04-04T17:30Z to 2016-04-05T20:36Z to https://tools.wmflabs.org/sal/production [production]
03:08 <l10nupdate@tin> ResourceLoader cache refresh completed at Thu Apr 7 03:08:28 UTC 2016 (duration 9m 52s) [production]
02:58 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.20) (duration: 10m 00s) [production]
02:38 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.19) (duration: 17m 51s) [production]
02:09 <chasemp> reboot db2047 (it's totally stuck w/ BUG: soft lockup - CPU#26 stuck for 22s! [migration/26:267] across) [production]
2016-04-06 §
23:44 <mattflaschen@tin> Synchronized php-1.27.0-wmf.19/extensions/Flow/includes/Data/Listener/NotificationListener.php: Fix new topic notifications (duration: 00m 29s) [production]
23:40 <mattflaschen@tin> Synchronized php-1.27.0-wmf.20/extensions/Flow/includes/Data/Listener/NotificationListener.php: Fix new topic notifications (duration: 00m 37s) [production]
23:11 <mattflaschen@tin> Synchronized docroot/wikipedia.org/apple-app-site-association: Support handoff and credential sharing with the iOS app (duration: 00m 34s) [production]
22:49 <bd808> Testing stashbot response in #wikimedia-fundraising [production]