651-700 of 6360 results (8ms)
2010-02-11 §
11:05 <RoanKattouw> Another load spike on the 8 CPU Apaches, approx 11:02-11:-05 UTC [production]
11:00 <RoanKattouw> Apache load spike started around 10:42 UTC and coincides with a load dip followed by a monitoring blackout on the Kennisnet Squids and a load spike on the Tampa Squids [production]
10:58 <RoanKattouw> 8CPU Apaches stopped going bonkers all of a sudden according to Ganglia, 4CPU ones still have high CPU usage [production]
10:56 <RoanKattouw> Site went down, all Apaches have high CPU usage [production]
2010-02-10 §
22:31 <catrope> synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix to deploy usability changes sitewide' [production]
22:22 <catrope> synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'Deploy r62275 to test' [production]
21:28 <catrope> synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'Deploy r62264 for real. Running svn up first helps' [production]
21:19 <catrope> synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'Deploy r62264' [production]
21:03 <andrew> synchronized php-1.5/wmf-config/InitialiseSettings.php [production]
21:02 <Andrew> Switching strategywiki to opt-out LiquidThreads, rather than opt-in [production]
20:46 <catrope> synchronized php-1.5/extensions/UsabilityInitiative/WikiEditor/WikiEditor.combined.min.js 'Deploy r62262' [production]
20:46 <catrope> synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'Deploy r62262' [production]
19:22 <catrope> synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' [production]
19:21 <catrope> synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'Deploying r62257' [production]
19:05 <Rob> storage1 disks replaced, except for the newly dead one, rma placed. [production]
18:39 <Rob> shutting down storage1 to replace its bad OS mirror disk [production]
16:52 <mark> Fixed MySQL permissions on srv186 as well [production]
16:15 <mark> Rigged puppet to deploy a PHP mail.ini file to the apaches, which sets it to call sendmail with -f <> (sending mail with empty envelope sender, like bounces) [production]
2010-02-09 §
22:06 <Andrew> LiquidThreads production software updates completed successfully. [production]
21:58 <andrew> synchronized php-1.5/extensions/LiquidThreads/classes/Dispatch.php [production]
21:57 <mark> Firewalled one ip on singer which was overloading Apache, looked like a Wordpress attack [production]
21:50 <mark> Lowered MaxClients setting to 250 on singer [production]
21:47 <Andrew> Running scap to deploy LiquidThreads updates [production]
21:45 <Andrew> Updating LiquidThreads production deployments to the alpha version in the next few minutes. [production]
21:44 <mark> Power cycled singer [production]
21:19 <mark> Fixed MySQL system user & permissions on srv151-srv185 [production]
21:17 <mark> Fixed MySQL instance on srv183 [production]
20:47 <mark> Fixed MySQL instance on srv156 [production]
19:39 <catrope> synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' [production]
19:29 <catrope> synchronized php-1.5/skins/common/edit.js 'Deploy r62190. Only for test for now, thank God for style versions' [production]
17:01 <catrope> synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' [production]
17:01 <catrope> synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'Deploying r62187' [production]
16:52 <catrope> synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' [production]
16:51 <catrope> synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'Deploying r62184' [production]
14:19 <catrope> synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' [production]
14:19 <catrope> synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'Deploying r62181' [production]
02:14 <andrew> synchronized php-1.5/extensions/LiquidThreads_alpha/lqt.js 'Merge r62158' [production]
2010-02-08 §
23:25 <catrope> synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix for r62041. Varnish needs this or Tampa will serve stale JS off bits' [production]
23:06 <mark> Deploying Exim on all application servers as well [production]
22:06 <catrope> synchronized php-1.5/extensions/UsabilityInitiative/OptIn/SpecialOptIn.php 'Deploy r62141' [production]
21:51 <mark> Added search3 and search9 to search_pool_1 on lvs3 [production]
21:48 <mark> Deploying Exim as a simple mail sending-only MTA (with queuing enabled) on squids, image scalers, search servers and core databases [production]
21:38 <Andrew> LiquidThreads wikinews rollout complete, though there was some buggage related to replication that had to be resolved. [production]
21:29 <andrew> synchronized php-1.5/extensions/LiquidThreads_alpha/classes/Dispatch.php [production]
21:29 <andrew> synchronized php-1.5/extensions/LiquidThreads_alpha/classes/Dispatch.php [production]
21:27 <andrew> synchronized php-1.5/extensions/LiquidThreads_alpha/classes/Dispatch.php [production]
21:24 <andrew> synchronized php-1.5/extensions/LiquidThreads_alpha/classes/Dispatch.php [production]
21:22 <andrew> synchronized php-1.5/extensions/LiquidThreads_alpha/classes/Dispatch.php [production]
21:21 <andrew> synchronized php-1.5/extensions/LiquidThreads_alpha/classes/Dispatch.php [production]
21:18 <andrew> synchronized php-1.5/wmf-config/InitialiseSettings.php [production]