|
2009-08-02
§
|
| 20:01 |
<mark> |
Started frontend squid on sq23 |
[production] |
| 16:12 |
<hcatlin> |
mobile1 ran out of disk b/c of log file size. Fixed now. |
[production] |
| 15:44 |
<mark> |
Eiximenis frontend seems limited at 26 k FDs somehow, reduced its LVS load again |
[production] |
| 14:23 |
<mark> |
Installed gmond on sq50 |
[production] |
| 14:09 |
<mark> |
Increased COSS cache dir sizes on sq50 from 15 to 35 GB, aufs from 20 to 24 GB |
[production] |
| 13:47 |
<mark> |
Increased LVS weight on eiximenis from 45 to 60 (4x other squids) |
[production] |
| 13:47 |
<mark> |
Increased COSS cache dir size on eiximenis from 65 GB to 128 GB |
[production] |
| 07:41 |
<brion> |
running parsertests on trunk checkins from r54052 on (not yet auto triggered) |
[production] |
| 07:20 |
<brion> |
synchronized php-1.5/wmf-config/CommonSettings.php |
[production] |
| 07:20 |
<brion> |
synchronized php-1.5/wmf-config/secret-projects.php |
[production] |
| 07:07 |
<brion> |
synchronized php-1.5/wmf-config/secret-projects.php 'add key for codereview test uploads' |
[production] |
| 07:04 |
<brion> |
updating CodeReview extension, starting test of the testing integration :) |
[production] |
|
2009-07-31
§
|
| 22:46 |
<tomaszf> |
running log splitter on searchidx1 to cleanup udplogger searchqueries |
[production] |
| 21:08 |
<Fred> |
bounced squid on sq23 |
[production] |
| 21:08 |
<Fred> |
bounced squid on sq43 |
[production] |
| 20:50 |
<Fred> |
and finally srv174 |
[production] |
| 20:49 |
<Fred> |
and srv32 / srv97 |
[production] |
| 20:48 |
<Fred> |
and srv207 |
[production] |
| 20:47 |
<Fred> |
bounced apache on srv129 |
[production] |
| 20:46 |
<Fred> |
added missing srv130 to apaches nodelist and synched common on it. |
[production] |
| 18:46 |
<Fred> |
re-synched nagios to get rid of 'false' mysql alerts |
[production] |
| 18:42 |
<Fred> |
deployed wikimedia-base (0.20) everywhere. |
[production] |
| 17:40 |
<Fred> |
updated wikimedia-base to include acct and enable SAR in order to do post-crash analysis when necessary. New version: 0.20. (Also changed dependency from ntp-simple to ntp as the ntp-simple package does not exist anymore) |
[production] |
| 15:17 |
<mark> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'Reenabled CentralNotice' |
[production] |
| 15:15 |
<kate> |
synchronized php-1.5/wmf-config/db-pmtpa.php 'remove db26 to dump s1 for TS' |
[production] |
| 14:15 |
<brion> |
removing stray readonly message from db.php on s3 (but not on s3frja... god our config's weird) |
[production] |
| 14:15 |
<brion> |
synchronized php-1.5/wmf-config/db-pmtpa.php |
[production] |
| 14:14 |
<brion> |
noting for toolserver repl fix: old s3 pos was db18-bin.090 454738665 |
[production] |
| 14:10 |
<brion> |
disabling general readonly |
[production] |
| 14:10 |
<brion> |
synchronized php-1.5/wmf-config/CommonSettings.php |
[production] |
| 14:09 |
<brion> |
s3 reset to master_host='db11', master_log_file='db11-bin.001', master_log_pos=79 |
[production] |
| 14:06 |
<brion> |
setting db11 up as temp master on s3 |
[production] |
| 14:04 |
<mark> |
Restarted MySQL on db18, running recovery |
[production] |
| 14:01 |
<brion> |
prepping a manual master switch |
[production] |
| 13:50 |
<brion> |
synchronized php-1.5/wmf-config/db-pmtpa.php 'disabled down db18' |
[production] |
| 13:49 |
<brion> |
removing down db18 |
[production] |
| 13:47 |
<brion> |
synchronized php-1.5/wmf-config/CommonSettings.php |
[production] |
| 13:28 |
<mark> |
synchronized php-1.5/wmf-config/db-pmtpa.php |
[production] |
| 13:27 |
<Andrew> |
Note that sync-file wmf-config/db-pmtpa.php does not work, you use sync-file db-pmtpa.php |
[production] |
| 13:27 |
<Andrew> |
Updated SwitchSettings.php for new location of db.php (wmf-config/db-pmtpa.php) |
[production] |
| 13:26 |
<andrew> |
synchronized php-1.5/wmf-config/db-pmtpa.php 'test' |
[production] |
| 12:40 |
<mark> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'temporarily disabled CentralNotice' |
[production] |
| 12:32 |
<Rob> |
scheduled reboot of csw5-pmtpa took place at 8am, traffic between esams and pmtpa is very high since as traffic and squid cache misses normalize out. |
[production] |