|
2012-10-15
§
|
| 19:57 |
<cmjohnson1> |
powering down srv266 troubleshooting per rt2896 |
[production] |
| 19:04 |
<binasher> |
restarted udp2log on oxygen with correct --recv-queue |
[production] |
| 18:58 |
<logmsgbot_> |
reedy synchronized wmf-config/ |
[production] |
| 18:55 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/includes/resourceloader/ResourceLoaderFileModule.php |
[production] |
| 18:49 |
<RoanKattouw> |
Created fluorine:/a/mw-log/udp2log/resourceloader.log chmod 644 and owned by udp2log:udp2log as root per Reedy's request |
[production] |
| 18:19 |
<logmsgbot_> |
reedy synchronized php-1.21wmf2/extensions/CodeReview |
[production] |
| 18:11 |
<logmsgbot_> |
reedy rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki and mediawikiwiki to 1.21wmf2 |
[production] |
| 18:07 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/extensions/MobileFrontend |
[production] |
| 17:32 |
<logmsgbot_> |
reedy synchronized wmf-config/CommonSettings.php 'wgDebugLogGroups for resourceloader' |
[production] |
| 17:30 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/includes/resourceloader/ResourceLoaderFileModule.php |
[production] |
| 16:28 |
<logmsgbot_> |
reedy synchronized php-1.21wmf2/extensions/MobileFrontend/ |
[production] |
| 16:26 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/extensions/MobileFrontend/ |
[production] |
| 16:16 |
<logmsgbot_> |
reedy synchronized wmf-config/InitialiseSettings.php 'Re-enable education program on enwiki' |
[production] |
| 16:06 |
<logmsgbot_> |
reedy Finished syncing Wikimedia installation... : Build localisation cache for 1.21wmf2 |
[production] |
| 15:43 |
<ottomata> |
stopping puppet on oxygen, looking for cause of recent packet loss |
[production] |
| 15:19 |
<logmsgbot_> |
reedy synchronized php-1.21wmf2/extensions/MobileFrontend |
[production] |
| 15:17 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/extensions/MobileFrontend |
[production] |
| 15:12 |
<logmsgbot_> |
reedy Started syncing Wikimedia installation... : Build localisation cache for 1.21wmf2 |
[production] |
| 14:56 |
<logmsgbot_> |
reedy rebuilt wikiversions.cdb and synchronized wikiversions files: test2wiki to 1.21wmf2 |
[production] |
| 14:55 |
<logmsgbot_> |
reedy synchronized wmf-config/ |
[production] |
| 14:52 |
<logmsgbot_> |
reedy synchronized php-1.21wmf2/ 'Initial sync' |
[production] |
| 13:35 |
<Reedy> |
Killing php-1.20wmf11 l10n cache from apaches |
[production] |
| 13:34 |
<Reedy> |
Killing php-1.20wmf11 l10n cache from apaches |
[production] |
| 13:31 |
<Reedy> |
Pointing php to php-1.21wmf1 |
[production] |
| 13:29 |
<logmsgbot_> |
reedy synchronized wmf-config/throttle.php 'Updated from gitrepo' |
[production] |
| 13:28 |
<logmsgbot_> |
reedy synchronized wmf-config/CommonSettings.php 'Remove extdist conditional for mount path' |
[production] |
| 11:49 |
<mark> |
Deployed range_offset_limit 64 MB change (from 0) to all upload backend squids |
[production] |
| 11:24 |
<paravoid> |
restarting pdns @ ns2 |
[production] |
| 11:16 |
<paravoid> |
switching European bits back to esams |
[production] |
| 09:49 |
<paravoid> |
restarting all imagescaler apaches, did not recover after swift outage |
[production] |
| 09:24 |
<paravoid> |
live-hacking swift on ms-be10, ms-be12, ms-be1 to remove fadvise calls |
[production] |
| 07:53 |
<Tim> |
increasing worker count to 100 on all swift backends, via puppet |
[production] |
| 07:37 |
<Tim> |
ms-be11 showed an immediate improvement in bandwidth out, but wchan still indicates that 48 is not enough, increasing to 100 |
[production] |
| 07:33 |
<Tim> |
experimentally doubling the worker count on ms-be11 since wchan indicates that the worker pool is exhausted |
[production] |
| 05:49 |
<apergos> |
ms-be12 reporting errors on disk /dev/sdi, I've rt'ed it an unmounted it in the meantime |
[production] |
| 04:34 |
<Tim> |
on ms-fe2: restarted swift-proxy due to mem leak |
[production] |
| 04:28 |
<Tim> |
on ms-fe3: restarting swift-proxy due to swap |
[production] |
| 04:26 |
<LeslieCarr> |
restarted swift proxy on ms-fe4 - it was in swap doom |
[production] |
| 02:25 |
<logmsgbot_> |
LocalisationUpdate completed (1.21wmf1) at Mon Oct 15 02:25:12 UTC 2012 |
[production] |