|
2011-03-08
ยง
|
| 21:18 |
<RoanKattouw> |
Restart updateCollation.php on s4 (commons) with a batch size of 50 to see how that plays out |
[production] |
| 21:07 |
<RoanKattouw> |
Move complete. Aborted updateCollation.php on fenari, now running 7 instances of /home/catrope/collatecluster in a screen on hume |
[production] |
| 21:01 |
<RoanKattouw> |
Breaking out updateCollation.php run into one process per DB cluster, and running them on hume rather than fenari |
[production] |
| 20:42 |
<catrope> |
synchronized php-1.17/includes/CategoryPage.php 'Experimental fix by Aryeh for category sorting problems' |
[production] |
| 20:35 |
<catrope> |
synchronized php-1.17/includes/parser/Parser.php |
[production] |
| 20:35 |
<catrope> |
synchronized php-1.17/includes/api/ApiQueryCategoryMembers.php |
[production] |
| 20:35 |
<catrope> |
synchronized php-1.17/includes/Title.php |
[production] |
| 20:34 |
<catrope> |
synchronized php-1.17/includes/LinksUpdate.php |
[production] |
| 20:34 |
<catrope> |
synchronized php-1.17/includes/CategoryPage.php |
[production] |
| 20:34 |
<catrope> |
synchronized php-1.17/extensions/intersection/DynamicPageList.php |
[production] |
| 20:34 |
<catrope> |
synchronized php-1.17/extensions/FlaggedRevs/specialpages/UnreviewedPages_body.php |
[production] |
| 20:34 |
<catrope> |
synchronized php-1.17/extensions/Collection/Collection.body.php |
[production] |
| 20:33 |
<catrope> |
synchronized php-1.17/extensions/CategoryTree/CategoryTreeFunctions.php |
[production] |
| 20:32 |
<RoanKattouw> |
Re-syncing a bunch of files without changes, didn't seem to have been picked up right by scap |
[production] |
| 19:38 |
<catrope> |
synchronized php-1.17/includes/api/ApiQueryCategoryMembers.php 'Sync without changes, maybe this didn't get picked up right' |
[production] |
| 19:35 |
<catrope> |
synchronized php-1.17/includes/CategoryPage.php 'Use array_merge instead of + for paranoia' |
[production] |
| 17:46 |
<RoanKattouw> |
Running updateCollation.php on all wikis |
[production] |
| 17:37 |
<RoanKattouw> |
Running updateCollation.php on cswiktionary |
[production] |
| 17:36 |
<robh> |
synchronizing Wikimedia installation... Revision: 83531: |
[production] |
| 17:24 |
<catrope> |
synchronizing Wikimedia installation... Revision: 83531: |
[production] |
| 17:23 |
<RoanKattouw> |
Deploying category collation code changes (r83254, r83531) sitewide using scap |
[production] |
| 15:16 |
<RoanKattouw> |
Updating MediaWiki on fenari from SVN to reintroduce category collation changes. For now, this is only for testing, so don't run sync-common-all or scap just yet |
[production] |
| 15:03 |
<RoanKattouw> |
Looks fine |
[production] |
| 15:02 |
<RoanKattouw> |
upgrade-1.17wmf1-final.php seems to have finished, running it again to verify it succeeded |
[production] |
| 13:28 |
<catrope> |
synchronized php-1.17/wmf-config/InitialiseSettings.php 'NamespaceWithSubpages settings for Czech wikis, per Danny B' |
[production] |
| 12:44 |
<mark> |
Changed 'wrr' scheduler to 'sh' on payments and owas LVS services |
[production] |
| 12:16 |
<midom> |
synchronized php-1.17/extensions/FlaggedRevs/specialpages/PendingChanges_body.php 'syncing http://p.defau.lt/?9RQEZQiZ_WeMT7_f7TpX6A live too' |
[production] |
| 12:09 |
<midom> |
synchronized php-1.17/extensions/FlaggedRevs/FlaggedRevs.hooks.php 'syncing livehack http://p.defau.lt/?BQwaKa6fH17X_CLt74khzA' |
[production] |
| 10:55 |
<Tim> |
running upgrade-1.17wmf1-final.php to upgrade old masters |
[production] |
| 10:53 |
<Tim> |
master positions: db36-bin.000001:106, db13-bin.000001:106, db31-bin.000001:106, db23-bin.001:79 |
[production] |
| 10:53 |
<tstarling> |
synchronized php-1.17/wmf-config/db.php |
[production] |
| 10:52 |
<tstarling> |
synchronized php-1.17/wmf-config/db.php |
[production] |
| 10:52 |
<Tim> |
switching master for s5: db14 -> db23 |
[production] |
| 10:52 |
<tstarling> |
synchronized php-1.17/wmf-config/db.php |
[production] |
| 10:52 |
<tstarling> |
synchronized php-1.17/wmf-config/db.php |
[production] |
| 10:51 |
<Tim> |
switching master for s4: db22 -> db31 |
[production] |
| 10:51 |
<tstarling> |
synchronized php-1.17/wmf-config/db.php |
[production] |
| 10:49 |
<tstarling> |
synchronized php-1.17/wmf-config/db.php |
[production] |
| 10:48 |
<Tim> |
switching master for s2: db30 -> db13 |
[production] |
| 10:48 |
<tstarling> |
synchronized php-1.17/wmf-config/db.php |
[production] |
| 10:48 |
<tstarling> |
synchronized php-1.17/wmf-config/db.php |
[production] |
| 10:48 |
<Tim> |
switching master for s1: db32 -> db36 |
[production] |
| 10:08 |
<midom> |
synchronized php-1.17/wmf-config/db.php 'adding db33 back with snapshotting disabled - checking if that plus temptables were causing the melt' |
[production] |
| 08:53 |
<jeluf> |
synchronized php-1.17/wmf-config/db.php 'removed db33 from s4' |
[production] |
| 08:51 |
<JeLuF> |
db33 down, commons unavailable |
[production] |
| 07:31 |
<tstarling> |
synchronized php-1.17/wmf-config/CommonSettings.php 're-enabled squid purging' |
[production] |
| 07:24 |
<tstarling> |
synchronized php-1.17/maintenance/jobs-loop.sh 'temp hack: no priority for htmlCacheUpdate' |
[production] |
| 07:22 |
<Tim> |
it appears more likely that the overload was due to htmlCacheUpdate jobs updating page_touched, in a flood after jobs-loop.sh fixes |
[production] |
| 07:18 |
<Tim> |
apaches overloaded after I fixed job queue squid purging. Disabled all squid purging and stopped job queue. |
[production] |
| 07:15 |
<tstarling> |
synchronized php-1.17/wmf-config/CommonSettings.php |
[production] |