|
2009-11-30
§
|
| 22:54 |
<Fred> |
rebooting pascal since it once again went down. |
[production] |
| 21:21 |
<RoanKattouw> |
Fred moved l10nupdate cronjob on hume to /etc/cron.d/l10nupdate , now runs as brion instead of root |
[production] |
| 21:21 |
<RoanKattouw> |
Unlocked /h/w/l10n/trunk/extensions , had been locked since Nov 19th blocking L10nUpdates to extensions |
[production] |
| 20:06 |
<rainman-sr> |
set search limit to 51 per Roan |
[production] |
| 20:06 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/EditToolbar/EditToolbar.php 'Raise style version for r59598, r59599' |
[production] |
| 20:05 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/EditToolbar/EditToolbar.js 'r59599 for real this time' |
[production] |
| 20:04 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/EditToolbar/EditToolbar.js 'r59599' |
[production] |
| 20:00 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/EditToolbar/EditToolbar.js 'r59598' |
[production] |
| 19:55 |
<rainman-sr> |
converting searchidx1 back to indexer, things seem to be back to normal now |
[production] |
| 19:46 |
<atglenn> |
enabled ExternalPages on test.wikipedia (fundraising extension) after adding it by hand to extension-list and ExtensionMessages.php |
[production] |
| 19:30 |
<rainman-sr> |
doing a delayed restart of all searchers |
[production] |
| 19:10 |
<mark> |
Setup puppet to enforce desired JVM alternatives on the search servers |
[production] |
| 17:00 |
<rainman-sr> |
all search servers (except search11) should be switched back from 64 bit java (java-6-openjdk) to 32 bit java (ia32-java-6-sun) |
[production] |
| 16:33 |
<rainman-sr> |
temporarely stop indexing and put searchidx1 in en.wp rotation, will back out when peak times are over |
[production] |
| 15:28 |
<mark> |
Puppet removed syslog-ng on nfs1/2 as well. Restored. |
[production] |
| 14:49 |
<mark> |
Replacing stock syslogd with rsyslog globally across the cluster (with puppet). rsyslog is the default in Ubuntu 9.10 onwards |
[production] |
| 10:35 |
<mark> |
Using streber for karmic install testing |
[production] |
| 10:35 |
<mark> |
Using brewster for karmic install testing |
[production] |
| 10:19 |
<mark> |
Created karmic-wikimedia APT repository |
[production] |
| 02:27 |
<rainman-sr> |
added search11 to en.wp search rotation.. we're getting peak-level traffic on 2am on sunday?! |
[production] |
|
2009-11-29
§
|
| 21:32 |
<mark> |
Unmounted /home on ms5 |
[production] |
| 20:58 |
<mark> |
Disabled NIS on ms5 |
[production] |
| 20:54 |
<mark> |
Carefully starting puppet management of ms5 (Solaris)... just ssh host key exchanging for now |
[production] |
| 20:42 |
<mark> |
Installed Blastwave puppet package (and deps) on ms5 |
[production] |
| 20:39 |
<mark> |
Installed Blastwave pkgutil on ms5 |
[production] |
| 20:37 |
<mark> |
Uninstalled coolstack CSKruntime and CSKruby packages on ms5 |
[production] |
| 20:16 |
<mark> |
Installed coolstack CSKruntime and CSKruby packages on ms5 |
[production] |
| 19:32 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 19:25 |
<domas> |
s3 replication switched from db11-bin.215:36889084 to db17-bin.001:79 |
[production] |
| 19:22 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 19:22 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 17:40 |
<mark> |
Fixed puppet config for searchidx1 |
[production] |
| 14:32 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'adding db1 and db7 to s6' |
[production] |
| 14:16 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Per Domas, do bug 21510 a little bit more performance-friendly' |
[production] |
| 14:12 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'bug 21510: Second attempt' |
[production] |
| 14:04 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'backing out last change, breaks' |
[production] |
| 14:03 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'bug 21510: Temp raise acct create throttle on eswiki for UEM IP for wiki week' |
[production] |
| 13:54 |
<mark> |
Installed puppet on db20, added new db20 node block in site.pp |
[production] |
| 13:37 |
<Tim> |
loaded arbcom election configuration into SecurePoll on enwiki |
[production] |
| 13:28 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'taking out db1 for db7 init' |
[production] |
| 13:19 |
<RoanKattouw> |
Running l10nupdate on hume for debugging |
[production] |
| 12:37 |
<domas> |
s6 start db29-bin.001:10246135 |
[production] |
| 12:32 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'welcome s6' |
[production] |
| 10:54 |
<domas> |
enwiki db is now served by 32g/16disk servers only... what is our next upgrade? :-) |
[production] |
| 10:53 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'taking out db7 from enwiki duty, it is too tiny...' |
[production] |
| 04:05 |
<tstarling> |
synchronized wmf-deployment/cache/trusted-xff.cdb |
[production] |
| 04:03 |
<Tim> |
updating TrustedXFF to r59540 |
[production] |
| 03:20 |
<Fred> |
distupgrade on filetserv1 (new kernel) |
[production] |
| 03:13 |
<Fred> |
apt-get upgrade on fileserv1... |
[production] |