|
2009-12-02
§
|
| 15:40 |
<Rob> |
rebooted the following per domas request: srv101 srv105 srv112 srv117 srv138 srv183 srv89 srv84 srv89 srv91 srv96 srv98 srv99 |
[production] |
| 15:02 |
<mark> |
Shutting down mint for installation of a wifi card |
[production] |
| 14:35 |
<Andrew> |
Scapping to update LiquidThreads alpha |
[production] |
| 14:34 |
<Andrew> |
Updates of LiquidThreads alpha to trunk state in progress |
[production] |
| 13:46 |
<RoanKattouw> |
Remove all LU cache files and rebuild from scratch to mitigate problems with root-owned cache files |
[production] |
| 13:41 |
<RoanKattouw> |
Purging LocalisationUpdate hashes and running the update script to ensure we didn't miss anything in the Nov 19 - Dec 1 blackout |
[production] |
| 05:28 |
<Tim> |
edited authz on mediawiki svn to deny write access to root |
[production] |
| 01:17 |
<tfinc> |
synchronized php-1.5/wmf-config/CommonSettings.php 'new banner for cn' |
[production] |
| 01:04 |
<Tim> |
restarted apache on srv211 and srv127 |
[production] |
|
2009-11-30
§
|
| 22:54 |
<Fred> |
rebooting pascal since it once again went down. |
[production] |
| 21:21 |
<RoanKattouw> |
Fred moved l10nupdate cronjob on hume to /etc/cron.d/l10nupdate , now runs as brion instead of root |
[production] |
| 21:21 |
<RoanKattouw> |
Unlocked /h/w/l10n/trunk/extensions , had been locked since Nov 19th blocking L10nUpdates to extensions |
[production] |
| 20:06 |
<rainman-sr> |
set search limit to 51 per Roan |
[production] |
| 20:06 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/EditToolbar/EditToolbar.php 'Raise style version for r59598, r59599' |
[production] |
| 20:05 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/EditToolbar/EditToolbar.js 'r59599 for real this time' |
[production] |
| 20:04 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/EditToolbar/EditToolbar.js 'r59599' |
[production] |
| 20:00 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/EditToolbar/EditToolbar.js 'r59598' |
[production] |
| 19:55 |
<rainman-sr> |
converting searchidx1 back to indexer, things seem to be back to normal now |
[production] |
| 19:46 |
<atglenn> |
enabled ExternalPages on test.wikipedia (fundraising extension) after adding it by hand to extension-list and ExtensionMessages.php |
[production] |
| 19:30 |
<rainman-sr> |
doing a delayed restart of all searchers |
[production] |
| 19:10 |
<mark> |
Setup puppet to enforce desired JVM alternatives on the search servers |
[production] |
| 17:00 |
<rainman-sr> |
all search servers (except search11) should be switched back from 64 bit java (java-6-openjdk) to 32 bit java (ia32-java-6-sun) |
[production] |
| 16:33 |
<rainman-sr> |
temporarely stop indexing and put searchidx1 in en.wp rotation, will back out when peak times are over |
[production] |
| 15:28 |
<mark> |
Puppet removed syslog-ng on nfs1/2 as well. Restored. |
[production] |
| 14:49 |
<mark> |
Replacing stock syslogd with rsyslog globally across the cluster (with puppet). rsyslog is the default in Ubuntu 9.10 onwards |
[production] |
| 10:35 |
<mark> |
Using streber for karmic install testing |
[production] |
| 10:35 |
<mark> |
Using brewster for karmic install testing |
[production] |
| 10:19 |
<mark> |
Created karmic-wikimedia APT repository |
[production] |
| 02:27 |
<rainman-sr> |
added search11 to en.wp search rotation.. we're getting peak-level traffic on 2am on sunday?! |
[production] |
|
2009-11-29
§
|
| 21:32 |
<mark> |
Unmounted /home on ms5 |
[production] |
| 20:58 |
<mark> |
Disabled NIS on ms5 |
[production] |
| 20:54 |
<mark> |
Carefully starting puppet management of ms5 (Solaris)... just ssh host key exchanging for now |
[production] |
| 20:42 |
<mark> |
Installed Blastwave puppet package (and deps) on ms5 |
[production] |
| 20:39 |
<mark> |
Installed Blastwave pkgutil on ms5 |
[production] |
| 20:37 |
<mark> |
Uninstalled coolstack CSKruntime and CSKruby packages on ms5 |
[production] |
| 20:16 |
<mark> |
Installed coolstack CSKruntime and CSKruby packages on ms5 |
[production] |
| 19:32 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 19:25 |
<domas> |
s3 replication switched from db11-bin.215:36889084 to db17-bin.001:79 |
[production] |
| 19:22 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 19:22 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 17:40 |
<mark> |
Fixed puppet config for searchidx1 |
[production] |
| 14:32 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'adding db1 and db7 to s6' |
[production] |
| 14:16 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Per Domas, do bug 21510 a little bit more performance-friendly' |
[production] |
| 14:12 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'bug 21510: Second attempt' |
[production] |
| 14:04 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'backing out last change, breaks' |
[production] |