351-400 of 10000 results (13ms)
2014-04-22 §
04:35 <paravoid> reactivate esams<->HE & eqiad<->HE peerings; issues are confirmed to be resolved [production]
03:32 <LocalisationUpdate> ResourceLoader cache refresh completed at Tue Apr 22 03:32:29 UTC 2014 (duration 32m 28s) [production]
02:43 <LocalisationUpdate> completed (1.24wmf1) at 2014-04-22 02:43:01+00:00 [production]
02:30 <LocalisationUpdate> completed (1.23wmf22) at 2014-04-22 02:29:34+00:00 [production]
01:16 <awight> nevermind previous update. [production]
01:11 <awight> update from 7dafce5f8fe265fb0ab6c96e01e59fc4362ea5b4 to cfe34fe0b10861167199a8f72bba279b9cac5e6e [production]
00:19 <catrope> synchronized php-1.24wmf1/extensions/MobileFrontend/javascripts/modules/editor/VisualEditorOverlay.js 'Fix JS error on save' [production]
00:15 <Reedy> torrus (on manutius) is down [production]
00:05 <RoanKattouw> Restarted ircecho on ekrem, IRC working again now [production]
00:00 <K4-713> updated payments from 2819549ab853ddd44c02519d4084580cb8572e7b --> 4811f6d3d80d126c8b3c89c11d20cc6416cb58f6 [production]
2014-04-21 §
23:57 <RoanKattouw> Started ircd on ekrem, startup doesn't seem to be puppetized [production]
23:24 <catrope> synchronized php-1.23wmf22/extensions/MultimediaViewer 'SWAT deploy cherry-picks' [production]
23:24 <catrope> synchronized php-1.24wmf1/extensions/MobileFrontend 'SWAT deploy cherry-picks' [production]
23:23 <catrope> synchronized php-1.24wmf1/extensions/MultimediaViewer 'SWAT deploy cherry-picks' [production]
23:23 <catrope> synchronized php-1.24wmf1/extensions/VisualEditor 'SWAT deploy cherry-picks' [production]
23:17 <catrope> synchronized wmf-config/InitialiseSettings.php 'Lithuanian namespace aliases for betawikiversity' [production]
23:11 <catrope> synchronized wmf-config/CommonSettings.php 'BetaFeatures whitelist' [production]
23:11 <catrope> synchronized wmf-config/InitialiseSettings.php 'Beta Features whitelist' [production]
22:51 <RoanKattouw> Restarted stuck Jenkins [production]
21:33 <awight> updated payments: af35b7b175bbe232 --> 2819549ab853ddd44c025 [production]
20:45 <andrewbogott> rebooted wtp1018 [production]
20:45 <subbu> deployed Parsoid ec51e5d1 (deploy SHA 0dd607fc) [production]
20:31 <manybubbles> rolling restart on remaining Elasticsearch servers to get the plugin (1010, 1011, 1012, 1015) [production]
19:45 <manybubbles> rolling restart on more of the elasticsearch servers to pick up plugins (06, 07, 09) [production]
19:15 <cmjohnson1> shutting down and relocating dobson [production]
19:15 <cmjohnson1> shutting down and relocating linne [production]
17:40 <cmjohnson> shutting down and relocating fenari [production]
17:37 <cmjohnson> shutting down mexia to relocate to 12th floor [production]
17:25 <aaron> synchronized php-1.24wmf1/includes/filerepo/file/LocalFile.php '2026e4aee7ffe00e0192914c0bd4a9ce04681c36' [production]
17:22 <aaron> synchronized wmf-config/PoolCounterSettings-eqiad.php 'Adjust large file download pool counter config to tie up less workers' [production]
17:18 <aaron> synchronized php-1.23wmf22/includes/filerepo/file/LocalFile.php '01ce2888d0d9011514a3fc16c1606b5b42f1ef37' [production]
17:13 <demon> synchronized wmf-config/InitialiseSettings.php 'renderfile-nonstandard throttle config' [production]
17:03 <aaron> synchronized php-1.24wmf1/thumb.php '44c46589ef02a30e60177ea268bc8de27a740434' [production]
17:02 <aaron> synchronized php-1.23wmf22/thumb.php '95913654cbf0a2c35ddbef74d135eeea71600d54' [production]
16:34 <ottomata> reinstalling elastic1013 (elastic1014 is still coming back online, but I don't want there to an extra eligible master for long) [production]
16:04 <ottomata> reinstalling elastic1014 [production]
15:22 <cmjohnson1> dataset2 going down to be relocated to the 12th floor [production]
13:41 <manybubbles> rolling restart on some of the Elasticsearch servers to pick up new plugins. should not cause any trouble. [production]
13:05 <Reedy> De-activated status.wm.o monitor for icinga due to false positive from HTTP auth [production]
12:54 <paravoid> demoting myself, removing Commons crat/admin rights [production]
12:41 <paravoid> escalating myself to Commons bureaucrat/admin, then adding GWToolset privileges [production]
12:40 <paravoid> deleting 29 GWToolset XML under Swift's wikipedia-commons-gwtoolset-metadata container for user Fæ/ [production]
11:51 <reedy> synchronized php-1.23wmf22/extensions/TimedMediaHandler 'I7483c8b7ec75f5149998da2b530ca04' [production]
11:51 <paravoid> deactivating esams<->HE peering, >90% packet loss between lon<->nyc [production]
11:49 <paravoid> deactivating eqiad<->HE peering, >90% packet loss between lon<->nyc [production]
11:45 <reedy> synchronized php-1.23wmf22/extensions/TimedMediaHandler 'I7483c8b7ec75f5149998da2b530ca0467ac70de7' [production]
03:55 <springle> reset pc100* slaves previously replicating from pmtpa [production]
03:33 <ori> 5.5k fatals over last 20 hrs, of which 3.5k are calls to doTransform() on a non-object at TimedMediaThumbnail.php:201, and 0.9k are Lua API OOMs at LuaSandbox/Engine.php:264 [production]
03:30 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Apr 21 03:30:15 UTC 2014 (duration 30m 14s) [production]
03:26 <ori> ap_busy_workers spike on image scalers eqiad, started ~2:55, subsided around ~3:20 [production]