|
2010-06-08
§
|
| 21:34 |
<Fred> |
mobile LVS setup. Traffic to mobile now balanced between mobile1 (30%) and mobile2 (70%) |
[production] |
| 19:58 |
<Rob> |
updated civicrm certificate file, had to apache2ctl restart on grosley |
[production] |
| 19:03 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Add .omniplan to extension whitelist for private wikis. Requested by Pete' |
[production] |
| 18:56 |
<Fred> |
new wikimedia-task-dns-auth deployed to nameservers. langlist regenerated. *lang*.m.wikipedia.org is now cnamed to m.wikimedia.org |
[production] |
| 16:41 |
<Rob> |
pushed updates to wordpress plugins on singer |
[production] |
| 16:40 |
<Fred> |
I foobared. need to redirect traffic to mobile2 so mobile1 can be re-imaged. |
[production] |
| 15:20 |
<Rob> |
updated singer and dns records for austin's setup of wm10reg |
[production] |
| 01:29 |
<Tim> |
running throughput test on mobile2 |
[production] |
|
2010-06-07
§
|
| 23:12 |
<Fred> |
modified puppet receipe for mobile to allow for easier passenger gem update. |
[production] |
| 22:04 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' |
[production] |
| 22:04 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/Vector/Vector.combined.min.js 'r67568' |
[production] |
| 20:55 |
<aaron> |
synchronized php-1.5/wmf-config/flaggedrevs.php 'rollback for de_labs editors' |
[production] |
| 19:46 |
<tfinc> |
synchronized php-1.5/extensions/CentralNotice/SpecialNoticeTemplate.php 'Picking up fix for tmp_id' |
[production] |
| 19:40 |
<catrope> |
synchronized php-1.5/extensions/CentralNotice/SpecialNoticeTemplate.php 'r67558' |
[production] |
| 19:12 |
<tfinc> |
synchronized php-1.5/extensions/CentralNotice/SpecialNoticeTemplate.php 'Picking up pagination from r67553' |
[production] |
| 18:56 |
<tfinc> |
synchronized php-1.5/extensions/CentralNotice/SpecialNoticeText.php 'Picking up pagination from r67553' |
[production] |
| 16:57 |
<Fred> |
Limiting message size to 5M for lists. |
[production] |
| 16:43 |
<Tim> |
moved mediawiki-cvs archive files away to prevent further archiving |
[production] |
| 16:26 |
<Tim> |
disabled archiving on problematic list mediawiki-cvs |
[production] |
| 14:07 |
<andrew> |
synchronized php-1.5/extensions/StrategyWiki/ActiveStrategy/ActiveStrategy_body.php |
[production] |
| 14:04 |
<andrew> |
synchronized php-1.5/extensions/StrategyWiki/ActiveStrategy/ActiveStrategy_body.php |
[production] |
| 13:49 |
<andrew> |
synchronized php-1.5/extensions/StrategyWiki/ActiveStrategy/ActiveStrategy_body.php |
[production] |
| 13:48 |
<andrew> |
synchronized php-1.5/extensions/StrategyWiki/ActiveStrategy/ActiveStrategy_body.php |
[production] |
| 13:46 |
<andrew> |
synchronized php-1.5/extensions/StrategyWiki/ActiveStrategy/ActiveStrategy_body.php |
[production] |
| 07:43 |
<Tim> |
on lily: fixed broken sources.list and upgraded python, installed python symbols |
[production] |
| 07:25 |
<Tim> |
on lily: restarted mailman (including ArchRunner) |
[production] |
| 07:07 |
<Tim> |
on lily: there are three instances of ArchRunner which are in a tight loop using CPU but not doing any syscalls, presumably dead. Renicing them, will try to make backtraces |
[production] |
| 03:41 |
<Tim> |
srv281 back up and resynced. However it crashed after being up only 3 days and shows more machine check errors. Suggest RMA. |
[production] |
| 03:25 |
<Fred> |
setup ganglia for lily. |
[production] |
| 03:07 |
<Tim> |
srv281 down for 41 hours, trying reboot |
[production] |
| 02:54 |
<Fred> |
upgraded gmond on lily. Modified gmond.conf to handle new modular arch. |
[production] |
| 02:46 |
<Tim> |
on lily: experimentally disabling broken bayes feature in spamassassin |
[production] |
| 02:35 |
<Tim> |
on lily: restarting spamd and fixing lock file errors |
[production] |
| 01:25 |
<Tim> |
on mobile1: Reduced PassengerMaxRequests to 5000 to reduce memory leakage |
[production] |
|
2010-06-06
§
|
| 11:33 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/UsabilityInitiative.hooks.php 'r67456' |
[production] |
| 11:33 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/css/combined.min.css 'r67456' |
[production] |
| 07:33 |
<Tim> |
on mobile1: use PassengerMaxRequests instead of PassengerPoolIdleTime, to avoid oscillation between 200 and 30 processes, once every half an hour due to unknown slowdown |
[production] |
| 03:10 |
<Tim> |
serve a 404 error for requests to the mobile server for domains other than *.m.wikipedia.org. DNS points here for lots of domains. |
[production] |
| 02:14 |
<Tim> |
installed a redirect from en.m.wikipedia.com to .org |
[production] |
| 01:18 |
<Tim> |
changed the mobile1 apache access log format to something more useful (and squid-like) |
[production] |
| 00:47 |
<Tim> |
on mobile1: installed logrotate script for apache2 (via puppet) |
[production] |