|
2011-09-27
§
|
| 22:00 |
<binasher> |
switching the S1 (enwiki) master |
[production] |
| 21:44 |
<binasher_> |
running 1.18 schema migrations on former s4 master (db31) |
[production] |
| 21:42 |
<binasher_> |
added srv171 back to dsh apache + mwinstall grps |
[production] |
| 21:38 |
<binasher_> |
new S4 master position - db22-bin.000040 516445005 |
[production] |
| 21:37 |
<asher> |
synchronized wmf-config/db.php 'writes on s4' |
[production] |
| 21:34 |
<asher> |
synchronized wmf-config/db.php 'new s4 master, setting read-only' |
[production] |
| 21:31 |
<binasher_> |
preparing to �swap masters on s4 (commonswiki) - will be read-only briefly |
[production] |
| 21:17 |
<aaron> |
synchronized multiversion/MWScript.php |
[production] |
| 20:59 |
<cmjohnson1> |
project2 rebooted per rt#1589 |
[production] |
| 20:14 |
<cmjohnson1> |
ms1 rebooted per rt#1577 |
[production] |
| 20:13 |
<robh> |
ms1 back online |
[production] |
| 20:05 |
<catrope> |
synchronized php-1.17/cache/interwiki.cdb '[[bugzilla:22407|bug 22407]] - Make mw: et al local interwiki prefixes' |
[production] |
| 19:57 |
<catrope> |
synchronized php-1.18/cache/interwiki.cdb 'Attempting to make mw: et al local interwiki prefixes' |
[production] |
| 19:48 |
<catrope> |
synchronized php-1.18/cache/interwiki.cdb 'Attempting to make mw: et al local interwiki prefixes' |
[production] |
| 19:40 |
<Ryan_Lane> |
rebooting gurvin |
[production] |
| 19:34 |
<Ryan_Lane> |
purged all ssl proxy configuration (and certs/keys) from yvon and gurvin |
[production] |
| 19:29 |
<catrope> |
synchronized php-1.18/includes/specials/SpecialUserlogin.php '[[rev:98238|r98238]]' |
[production] |
| 19:26 |
<notpeter> |
restarting nagios on spence |
[production] |
| 19:21 |
<Jeff_Green> |
revised apache config puppetized and installed on grosley |
[production] |
| 19:17 |
<Ryan_Lane> |
repooling gurvin |
[production] |
| 19:14 |
<Ryan_Lane> |
depooling gurvin and yvon, and pooling ssl2 and ssl3 |
[production] |
| 19:05 |
<Ryan_Lane> |
depooling ssl1-4 and repooling gurvin and yvon |
[production] |
| 19:01 |
<Ryan_Lane> |
depooling gurvin for testing |
[production] |
| 18:55 |
<Ryan_Lane> |
depooling yvon, and adding ssl1-4 to https lvs group |
[production] |
| 18:52 |
<Ryan_Lane> |
pushed version of nginx with udplogging module to repo |
[production] |
| 18:25 |
<notpeter> |
restarting srv159 and srv189. swapdeath. |
[production] |
| 17:58 |
<RoanKattouw> |
Fixing corrupted article_feedback_revisions rows by hand in the enwiki DB |
[production] |
| 17:56 |
<apergos> |
restarted lighthttpd on dataset2, it had stopped |
[production] |
| 17:37 |
<Ryan_Lane> |
installing ssl1-4 |
[production] |
| 17:35 |
<py> |
synchronized wmf-config/mc.php 'putting mw24-mw27 into mc.php as memcache spares as part of RT 1550' |
[production] |
| 17:27 |
<catrope> |
synchronized php-1.17/extensions/ArticleFeedback/api/ApiArticleFeedback.php '[[rev:98232|r98232]]' |
[production] |
| 17:05 |
<Jeff_Green> |
stopping mysql on db1008 and cloning to db1048 |
[production] |
| 16:22 |
<catrope> |
synchronized php-1.18/resources/startup.js 'touch' |
[production] |
| 16:20 |
<catrope> |
synchronized wmf-config/CommonSettings.php 'Temporarily set $wgResourceLoaderValidateJS to false for [[bugzilla:31187|bug 31187]]' |
[production] |
| 16:20 |
<notpeter> |
stopping jobrunners on all external store apaches and removing from puppet/dsh group |
[production] |
| 15:56 |
<Jeff_Green> |
throwing payments4 back in the pool to test new firewall rules |
[production] |
| 15:22 |
<robh> |
updated dns for ssl1-ssl4 |
[production] |
| 15:05 |
<py> |
synchronized wmf-config/db.php 'putting srv180 and srv163 back into db.php as they are back up' |
[production] |
| 14:34 |
<notpeter> |
restarting srv180 |
[production] |
| 14:33 |
<notpeter> |
restarting srv163 (this time the right one...) |
[production] |
| 14:18 |
<ariel> |
synchronized php-1.18/includes/HistoryBlob.php 'mhash workaround from 117wmf1' |
[production] |
| 14:11 |
<notpeter> |
restarting srv161 |
[production] |
| 13:56 |
<notpeter> |
powercycling srv188 |
[production] |
| 13:54 |
<RoanKattouw> |
Fixed ownership of /apache/common/php-1.17/cache/l10n from nagios:wikidev to mwdeploy:mwdeploy |
[production] |
| 13:38 |
<notpeter> |
restarting srv171 |
[production] |
| 13:37 |
<catrope> |
synchronized wmf-config/db.php 'Comment out srv163 and srv180 from ES list, they are dead' |
[production] |
| 13:12 |
<notpeter> |
powercycled srv179 to bring it back up |
[production] |
| 12:19 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Bug 31172 - Deploy WikiLove on orwiki' |
[production] |
| 12:14 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Bug 31178 - Enable wikilove in Portuguese Wikipedia' |
[production] |
| 11:36 |
<RoanKattouw> |
Edited /home/wikipedia/bin/sync-common-file and added -o -oSetupTimeout=10 to the dsh command line. This makes sync-file not hang forever when an Apache is down. Thanks to maplebed for the tip |
[production] |