|
2010-02-02
§
|
| 23:26 |
<mark> |
PyBal got into a confused state due to a duplicate LVS realserver entry. Set up LVS manually/statically for 30 mins to debug the problem. PyBal is now active again. |
[production] |
| 23:04 |
<Tim> |
pmtpa text squids down |
[production] |
| 22:33 |
<mark> |
Pooled sq59-68 in LVS (Text) |
[production] |
| 22:27 |
<apergos> |
restarted apache on srv170 (corrupted apc cache) |
[production] |
| 22:25 |
<mark> |
Increased CARP weight of the new Upload squids sq51-58 from 10 to 30 |
[production] |
| 22:17 |
<mark> |
Added sq59-66 to the Text pool of Squids, with low CARP weight (10) to seed the caches |
[production] |
| 21:44 |
<andrew> |
synchronized php-1.5/wmf-config/liquidthreads.php |
[production] |
| 21:42 |
<Andrew> |
temporarily disabling $wgLqtEnotif to mitigate mail server DoS impact on LiquidThreads post/reply speed. |
[production] |
| 21:42 |
<andrew> |
synchronized php-1.5/extensions/LiquidThreads_alpha/classes/NewMessagesController.php 'Deploy r61879' |
[production] |
| 21:42 |
<andrew> |
synchronized php-1.5/extensions/LiquidThreads/classes/NewMessagesController.php 'Deploy r61879' |
[production] |
| 21:04 |
<mark> |
Removed sq2-10 from the LVS pool |
[production] |
| 20:46 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version addition for usability alpha' |
[production] |
| 20:45 |
<catrope> |
synchronized php-1.5/skins/common/jquery.min.js 'Deploing r61700' |
[production] |
| 20:38 |
<mark> |
Added sq52-58 to the backend Squid pool with low CARP weight, to seed the caches |
[production] |
| 20:37 |
<catrope> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'bug 22351: Enable Vector and toolbar on srwikinews' |
[production] |
| 20:36 |
<mark> |
Increased Exim load average threshold for delivery on mchenry |
[production] |
| 19:09 |
<Fred> |
rebooted singer (went down for unknown- at this time- reasons). |
[production] |
| 18:25 |
<Fred> |
OS-installed sq52-ssq58 |
[production] |
| 18:14 |
<mark> |
OS-installed sq59-sq66 |
[production] |
| 17:23 |
<mark> |
Thrown sq66 backend squid in the Squid pool, CARP weight 30 |
[production] |
| 06:01 |
<Tim> |
deploying r61846 via scap |
[production] |
| 04:25 |
<Andrew> |
Schema change successfully applied. |
[production] |
| 04:24 |
<Andrew> |
Applying schema change thread_signature.sql to mediawikiwiki, strategywiki, testwiki |
[production] |
| 04:22 |
<Andrew> |
Applying schema change thread_signature.sql to liquidthreads_labswikimedia |
[production] |
| 01:31 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Fix for UsabilityInitiative_alpha config' |
[production] |
| 01:18 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative_alpha/WikiEditor/WikiEditor.combined.min.js 'r61842' |
[production] |
| 01:13 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Append to $wgStyleVersion when running usability alpha' |
[production] |
| 01:10 |
<catrope> |
synchronized php-1.5/includes/OutputPage.php 'r61841 for real this time' |
[production] |
| 01:09 |
<catrope> |
ran sync-common-all |
[production] |
| 01:07 |
<catrope> |
synchronized php-1.5/includes/OutputPage.php 'r61841' |
[production] |
| 01:05 |
<catrope> |
synchronized php-1.5/includes/OutputPage.php 'r61839' |
[production] |
| 00:51 |
<RoanKattouw> |
Running scap |
[production] |
| 00:48 |
<catrope> |
synchronized php-1.5/includes/OutputPage.php 'r61837' |
[production] |
| 00:40 |
<RoanKattouw> |
Synced extensions/UsabilityInitiative_alpha by hand |
[production] |
| 00:07 |
<catrope> |
synchronized php-1.5/skins/common/jquery.min.js |
[production] |
| 00:07 |
<RoanKattouw> |
Syncing some CSS and JS files for usability deployment so bits.wikimedia.org will pick them up |
[production] |
| 00:06 |
<catrope> |
synchronized php-1.5/skins/common/wikibits.js 'Sync changes to wikibits.js in r61558' |
[production] |
|
2010-02-01
§
|
| 23:30 |
<RoanKattouw> |
Testing new usability code on test |
[production] |
| 23:27 |
<mark> |
Pooled sq51 frontend squid in the upload LVS cluster, weight 30 |
[production] |
| 23:03 |
<mark> |
Thrown sq51 backend squid in the Squid pool, CARP weight 30 |
[production] |
| 20:50 |
<Rob> |
srv217 shutdown until checked |
[production] |
| 20:50 |
<Rob> |
srv217 not responsive to network, but LOM works and system is jut not working online, possible bad cable, will check in DC tomorrow |
[production] |
| 20:43 |
<mark> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Added 36 new Squid servers, removed bart' |
[production] |
| 20:39 |
<mark> |
Deployed new wikimedia-base package that no longer manages sysctl.conf post Hardy, letting puppet handle it |
[production] |
| 20:10 |
<atglenn> |
restarted webserver on ms7 to read new conf (fix bug 22321) |
[production] |
| 20:09 |
<mark> |
Granted cary access to the OTRS db from fenari |
[production] |
| 19:27 |
<RoanKattouw> |
srv9 reported down, pinging yields Host Unreachable. srv9 is not in Nagios but seems to be in use |
[production] |
| 18:51 |
<Fred> |
swapped memcache node from srv145 to srv196 |
[production] |