|
2012-02-28
§
|
| 04:48 |
<Tim> |
on manutius: rebuilt torrus config DB, moved old one out to /var/lib/torrus/db.broken. Restarted. |
[production] |
| 04:30 |
<Tim> |
torrus down, reporting "PANIC: fatal region error detected; run recovery" about its DB files, will stop apache to investigate |
[production] |
| 03:54 |
<tstarling> |
synchronized deleted.dblist 'removed chwikimedia' |
[production] |
| 03:54 |
<tstarling> |
synchronized all.dblist 'removed chwikimedia' |
[production] |
| 03:07 |
<tstarling> |
synchronized php-1.19/includes/specials/SpecialMovepage.php '[[rev:112572|r112572]]' |
[production] |
| 02:59 |
<catrope> |
synchronized php-1.19/resources '[[rev:112570|r112570]]' |
[production] |
| 02:59 |
<catrope> |
synchronized php-1.19/extensions/CheckUser '[[rev:112570|r112570]]' |
[production] |
| 02:58 |
<catrope> |
synchronized php-1.19/includes/logging '[[rev:112570|r112570]]' |
[production] |
| 02:44 |
<aaron> |
synchronized php-1.19/includes/Block.php 'deployed [[rev:112564|r112564]]' |
[production] |
| 02:43 |
<aaron> |
synchronized php-1.19/includes/resourceloader/ResourceLoaderContext.php 'deployed [[rev:112564|r112564]]' |
[production] |
| 02:43 |
<aaron> |
synchronized php-1.19/resources/mediawiki/mediawiki.js 'deployed [[rev:112564|r112564]]' |
[production] |
| 02:33 |
<LocalisationUpdate> |
completed (1.19) at Tue Feb 28 02:33:30 UTC 2012 |
[production] |
| 02:17 |
<LocalisationUpdate> |
completed (1.18) at Tue Feb 28 02:17:09 UTC 2012 |
[production] |
| 02:13 |
<binasher> |
moved db1006 to new s6 master |
[production] |
| 01:56 |
<Tim> |
reduced disk space usage on srv191 by running logrotate manually |
[production] |
| 00:43 |
<tstarling> |
rebuilt wikiversions.cdb and synchronized wikiversions files: plwiki to 1.19 |
[production] |
| 00:35 |
<aaron> |
synchronized php-1.19/includes/StubObject.php 'trigger errors for debugging bad callbacks' |
[production] |
| 00:28 |
<Tim> |
removed 2GB of syslogs on srv192 |
[production] |
| 00:25 |
<tstarling> |
synchronized php-1.18/includes/resourceloader/ResourceLoaderFileModule.php |
[production] |
| 00:21 |
<tstarling> |
synchronized php-1.19/includes/api/ApiQueryLogEvents.php 'revert live hack' |
[production] |
| 00:20 |
<tstarling> |
synchronized php-1.19/includes/api/ApiQueryRecentChanges.php |
[production] |
|
2012-02-27
§
|
| 23:46 |
<catrope> |
synchronized php-1.19/includes/api/ApiQueryLogEvents.php 'Attempted bugfix' |
[production] |
| 23:17 |
<aaron> |
rebuilt wikiversions.cdb and synchronized wikiversions files: nlwiki to 1.19 |
[production] |
| 22:54 |
<reedy> |
synchronized php-1.19/includes/MessageBlobStore.php '[[rev:112536|r112536]]' |
[production] |
| 22:45 |
<RobH> |
dns update for manganese server |
[production] |
| 22:33 |
<reedy> |
synchronized php-1.19/includes/ '[[rev:112532|r112532]]' |
[production] |
| 22:31 |
<binasher> |
new s4 master pos - MASTER_LOG_FILE='db31-bin.000253', MASTER_LOG_POS=457980068 |
[production] |
| 22:30 |
<asher> |
synchronized wmf-config/db.php 'done s4 switch' |
[production] |
| 22:29 |
<binasher> |
switching s4 master to db31 |
[production] |
| 22:29 |
<asher> |
synchronized wmf-config/db.php 'switching s4 master to db31, setting read-only' |
[production] |
| 22:26 |
<reedy> |
synchronized php-1.19/extensions/UploadWizard/UploadWizard.i18n.php '[[rev:112531|r112531]]' |
[production] |
| 22:16 |
<RobH> |
cadmium locked up, rebooting |
[production] |
| 22:14 |
<binasher> |
running 1.19 schema migration script to get former s5, s6, s1 masters (db45, db47, db36) |
[production] |
| 22:09 |
<binasher> |
new s1 (enwiki) master pos - MASTER_LOG_FILE='db38-bin.000129', MASTER_LOG_POS=255719721 |
[production] |
| 21:56 |
<asher> |
synchronized wmf-config/db.php 'done s1 switch' |
[production] |
| 21:55 |
<asher> |
synchronized wmf-config/db.php 'swapping s1 enwiki master to db38, setting read-only' |
[production] |
| 21:53 |
<binasher> |
preparing to swap enwiki master, it will be read only for a couple minutes |
[production] |
| 21:52 |
<catrope> |
synchronized php-1.19/extensions/ProofreadPage/proofread.js '[[rev:112522|r112522]]' |
[production] |
| 21:49 |
<catrope> |
synchronized php-1.19/extensions/ProofreadPage/proofread.js '[[rev:112522|r112522]]' |
[production] |
| 21:15 |
<nikerabbit> |
synchronized php-1.18/resources/startup.js 'touch' |
[production] |
| 21:02 |
<binasher> |
db1006 (s6-secondary) is still slaving from db47 - it's very behind post hw failure. need to manually swap to db43 once caught up |
[production] |
| 21:02 |
<binasher> |
new s6 master pos - MASTER_LOG_FILE='db43-bin.000027', MASTER_LOG_POS=577074024 |
[production] |
| 21:01 |
<asher> |
synchronized wmf-config/db.php 'done s6 master swap' |
[production] |
| 20:59 |
<asher> |
synchronized wmf-config/db.php 'swapping s6 master to db43, setting read-only' |
[production] |
| 20:59 |
<binasher> |
preparing to switch s6 master |
[production] |
| 20:42 |
<binasher> |
powercycled db1006 after finding nothing on the serial console. booted without issue, then started mysql. |
[production] |
| 20:41 |
<hashar> |
synchronized php-1.19/extensions/CodeReview/backend/DiffHighlighter.php 'Special:CodeReview merge [[rev:112513|r112513]] fix [[bugzilla:27375|bug 27375]]' |
[production] |
| 20:31 |
<binasher> |
new s5 master pos - MASTER_LOG_FILE='db35-bin.000011', MASTER_LOG_POS=374074061 |
[production] |
| 20:27 |
<asher> |
synchronized wmf-config/db.php 'done s5 master swap' |
[production] |
| 20:25 |
<asher> |
synchronized wmf-config/db.php 'swapping s5 master to db35, setting read-only' |
[production] |