|
2011-07-20
§
|
| 21:53 |
<Ryan_Lane> |
added monitoring for esams and pmtpa -lb addresses |
[production] |
| 21:10 |
<Jeff_Green> |
db10 mysqldump is done, replication restarted |
[production] |
| 21:05 |
<Ryan_Lane> |
adding lvs IPs for all esams -lb addresses to amslvs servers |
[production] |
| 20:54 |
<catrope> |
synchronized php-1.17/resources/startup.js 'touch' |
[production] |
| 20:52 |
<catrope> |
synchronized wmf-config/InitialiseSettings.php 'Raise AFT percentage to 70%' |
[production] |
| 20:52 |
<catrope> |
synchronized wmf-config/CommonSettings.php 'Lower AFT clicktracking to 0.386%' |
[production] |
| 20:47 |
<Ryan_Lane> |
added all new -lb services to esams pybal |
[production] |
| 20:45 |
<Ryan_Lane> |
made esams pybal files consistent. restarting pybal on each amslvs system, in order |
[production] |
| 20:39 |
<catrope> |
synchronized php-1.17/resources/startup.js 'touch' |
[production] |
| 20:37 |
<catrope> |
synchronized wmf-config/InitialiseSettings.php 'Raise AFT percentage to 63%' |
[production] |
| 20:37 |
<catrope> |
synchronized wmf-config/CommonSettings.php 'Lower AFT clicktracking percentage to 0.428571%' |
[production] |
| 20:25 |
<catrope> |
synchronized php-1.17/resources/startup.js 'touch' |
[production] |
| 20:24 |
<catrope> |
synchronized wmf-config/InitialiseSettings.php 'Raise AFT percentage to 57%' |
[production] |
| 20:22 |
<catrope> |
synchronized wmf-config/CommonSettings.php 'Lower AFT clicktracking rate to 0.47%' |
[production] |
| 19:02 |
<awjrichards> |
synchronized php/extensions/ContactPageFundraiser/SpecialContact.php '[[rev:92653|r92653]]' |
[production] |
| 19:02 |
<awjrichards> |
synchronized php/extensions/ContactPageFundraiser/README '[[rev:92653|r92653]]' |
[production] |
| 19:02 |
<awjrichards> |
synchronized php/extensions/ContactPageFundraiser/install.settings '[[rev:92653|r92653]]' |
[production] |
| 19:01 |
<awjrichards> |
synchronized php/extensions/ContactPageFundraiser/ContactPage.i18n.php '[[rev:92653|r92653]]' |
[production] |
| 19:01 |
<awjrichards> |
synchronized php/extensions/ContactPageFundraiser/ContactPage.alias.php '[[rev:92653|r92653]]' |
[production] |
| 18:59 |
<awjrichards> |
synchronized wmf-config/CommonSettings.php 'Variable-izing enabling of ContactPageFundraiser extension, enabling on testwiki and foundation wiki' |
[production] |
| 18:56 |
<awjrichards> |
synchronized wmf-config/InitialiseSettings.php 'Variable-izing enabling of ContactPageFundraiser extension' |
[production] |
| 18:54 |
<aaron> |
synchronized wmf-config/PrivateSettings.php |
[production] |
| 18:42 |
<AaronSchulz> |
Changed sync-file to be relative to common/ instead of php-1.17/ |
[production] |
| 18:41 |
<aaron> |
synchronizing Wikimedia installation... Revision: 92561: |
[production] |
| 18:41 |
<RobH> |
rebooting the offline sq63 to poll its SAS controller info |
[production] |
| 18:25 |
<Ryan_Lane> |
restarting pybal on lvs1 |
[production] |
| 18:12 |
<Jeff_Green> |
stopping replication on db10 and redoing dump with mysqldump instead of mydumper... |
[production] |
| 18:03 |
<awjrichards> |
synchronized php-1.17/extensions/LandingCheck/SpecialLandingCheck.php '[[rev:92647|r92647]]' |
[production] |
| 18:03 |
<awjrichards> |
synchronized php-1.17/extensions/LandingCheck/LandingCheck.i18n.php '[[rev:92647|r92647]]' |
[production] |
| 17:40 |
<RobH> |
lvs3 racked, but powered off. mgmt confirmed working, network port confirmed up |
[production] |
| 16:09 |
<mark> |
Disabling RunCommand monitor on lvs4; not working right |
[production] |
| 15:50 |
<mark> |
Restarted PyBal on lvs4 |
[production] |
| 15:27 |
<RobH> |
powercycling srv283 |
[production] |
| 15:27 |
<RobH> |
srv283 is hung up again, and now wont take my commands in serial console... |
[production] |
| 15:22 |
<RobH> |
srv283 loads are insane, its very slugging, and only serial console works. kicking the apache process over to free up memory and resume normal function |
[production] |
| 14:39 |
<RobH> |
srv239 unresponsive, rebooting |
[production] |
| 14:02 |
<mark> |
Setup nas1 with clustering failover |
[production] |
| 08:44 |
<mark> |
Fixed misc::mediawiki-logger manifest |
[production] |
| 00:04 |
<Ryan_Lane> |
test [[bugzilla:1|bug 1]] |
[production] |
|
2011-07-19
§
|
| 23:59 |
<binasher> |
also set long_query_time to 0.8 on the enwiki read slaves |
[production] |
| 23:09 |
<Ryan_Lane> |
added new ganglia clusters for ssl cluster in pmtpa and esams |
[production] |
| 22:50 |
<binasher> |
reduced long_query_time on db36 to 800ms |
[production] |
| 22:46 |
<binasher> |
setting up mk-query-digest slow query processing on db36 |
[production] |
| 22:43 |
<binasher> |
installing maatkit on core dbs via puppet |
[production] |
| 22:13 |
<Jeff_Green> |
installing mysql 5.0.51a on db1025 to for mydumper/myloader mysqld version upgrade testing |
[production] |
| 21:40 |
<Ryan_Lane> |
make that changing the CNAME to wikipedia-lb |
[production] |
| 21:40 |
<Ryan_Lane> |
changing DNS entry for test.wikipedia.org to wikipedia-lb.wikimedia.org |
[production] |
| 21:04 |
<catrope> |
synchronized php-1.17/resources/startup.js 'touch' |
[production] |
| 21:03 |
<catrope> |
synchronized wmf-config/CommonSettings.php 'Kill underscores in blacklist category name' |
[production] |
| 20:58 |
<catrope> |
synchronized php-1.17/resources/startup.js 'touch' |
[production] |