551-600 of 8442 results (8ms)
2010-08-11 §
11:19 <tstarling> synchronized php-1.5/includes/media/Bitmap.php 'reduced magick memory limit from 100M to 50M to stop hanging with vsize limit 300M' [production]
10:46 <mark> Removed pattern check from nagios check_http [production]
09:42 <tstarling> synchronized php-1.5/wmf-config/CommonSettings.php [production]
09:38 <tstarling> synchronized php-1.5/wmf-config/CommonSettings.php [production]
09:35 <Tim> rebooting srv223, went OOM and mostly died [production]
09:32 <tstarling> synchronized php-1.5/includes/media/Bitmap.php 'temporary patch to stop scalers going OOM' [production]
09:19 <Tim> temporarily increased memory limit on the image scalers, since the new convert tends to hang when it runs out of memory instead of crashing nicely [production]
09:17 <tstarling> synchronized php-1.5/wmf-config/CommonSettings.php 'more memory for image scalers' [production]
08:56 <Tim> upgrading imagemagick on image scalers to 6.6.2.6-1wm1, package recently committed to svn [production]
02:48 <Tim> on techblog, disabled WP_DEBUG since it was messing up the admin panels with E_NOTICE messages [production]
02:42 <Tim> disabled WP-SpamFree on techblog due to bug 19540 [production]
2010-08-10 §
23:12 <Fred> upgraded Tridge to Lucid. Now rebooting. [production]
22:04 <RobH> knsq10 back online [production]
20:59 <RobH> knsq10 reinstalling [production]
20:44 <RobH> knsq9 online [production]
19:37 <RobH> handed off knsq8 to mark, reinstalling knsq9 [production]
19:02 <^demon> disabled svn post-commit hook for parser tests, long-since broken [production]
18:57 <mark> Stopping backend squid on amssq60 for testing [production]
15:24 <RobH> knsq8 reinstalled, not yet online, will push online shortly [production]
14:56 <mark> Setup RT on rt.wikimedia.org (streber) [production]
14:32 <RobH> knsq30 online and in cluster, knsq8 coming down for work [production]
14:18 <RobH> updated wordpress versions on blog.wikimedia.org and techblog.wikimedia.org [production]
13:35 <RobH> finishing install on knsq30 [production]
12:50 <Tim> installed schroot on stafford, for hardy versions of uupdate etc. [production]
11:19 <mark> Fixed broken hourly cron job mw-serve [production]
11:18 <mark> Changed su www-data into su mwlib in cleanup cronjob on pdf1 [production]
10:23 <mark> Removed broken daily system health report on srv178 [production]
10:22 <mark> Removed broken daily system health report on db4 [production]
07:13 <andrew> synchronized php-1.5/extensions/CommunityApplications/SpecialCommunityApplications.php 'Merge r70798' [production]
07:13 <andrew> synchronized php-1.5/extensions/CommunityApplications/CommunityApplications.i18n.php 'Merge r70798' [production]
04:02 <RobH> knsq30 set to false in pybal, install half done, will finish tomorrow morning. [production]
02:44 <RobH> knsq29 online and in cluster [production]
02:30 <RobH> knsq30 reinstalling [production]
00:09 <RobH> knsq28 back online [production]
00:03 <RobH> knsq27 back online [production]
00:03 <RobH> knsq29 reinstalling [production]
2010-08-09 §
23:34 <RobH> knsq28 reinstalling [production]
23:32 <RobH> knsq26 online [production]
23:32 <RobH> knsq25 online [production]
23:12 <RobH> continuing reinstallation, ignore errors for knsq27, reinstalling [production]
22:33 <RobH> knsq23, knsq24 back online, knsq25, knsq26 still being reinstalled, knsq27-30 still online not yet reinstalled [production]
21:51 <mark> Added Nagios router interfaces check for br1-knams (using puppet) [production]
21:11 <mark> Unmounted /dev/sda6 (/a) on srv171, replaced it by /dev/mapper/nonredundant-data (LV with the same data and more space) [production]
21:02 <RobH> knsq24, knsq25, knsq26, knsq27 coming down for reinstall and puppetfication [production]
20:49 <RobH> knsq23 reinstall done and pushed back into cluster [production]
18:40 <mark> Running apt-get upgrade on db9 [production]
18:29 <mark> Fixed ganglia mess on sq45 [production]
18:24 <mark> Powercycled sq45 [production]
18:18 <mark> Added a new MegaCli64 to wikimedia-raid-utils, made check-raid.py use it instead (we have all 64 bit servers anyway), and deployed the new package to the repository. Puppet will upgrade it everywhere. [production]
16:26 <Fred> fixed DPKG issue on transcode... another one of those conflicting gmond install [production]