|
2011-02-08
§
|
| 16:32 |
<RoanKattouw> |
Running scap |
[production] |
| 16:31 |
<RoanKattouw> |
Switching cluster to 1.17 |
[production] |
| 16:30 |
<RoanKattouw> |
Switching /home to 1.17 |
[production] |
| 16:29 |
<catrope> |
ran sync-common-all |
[production] |
| 16:29 |
<RoanKattouw> |
Running sync-common-all to push changes to php-1.17 |
[production] |
| 16:28 |
<RoanKattouw> |
Switching test back to 1.16 in preparation for sync-common-all |
[production] |
| 15:13 |
<RoanKattouw> |
test is now running 1.17wmf1 again |
[production] |
| 14:42 |
<RoanKattouw> |
Forgot to log that I put testwiki back on 1.16 about 30 mins ago |
[production] |
| 14:29 |
<catrope> |
synchronized php-1.5/languages/messages/MessagesJa.php 'Testing to see if this will fix the message caching issue' |
[production] |
| 14:00 |
<RoanKattouw> |
Moving directories on /home again, putting 1.17 on test so we can diagnose the cause of the overload |
[production] |
| 13:41 |
<catrope> |
synchronizing Wikimedia installation... Revision: 81334: |
[production] |
| 13:41 |
<RoanKattouw> |
Running scap |
[production] |
| 13:40 |
<RoanKattouw> |
1.17-rollback done |
[production] |
| 13:39 |
<RoanKattouw> |
Running 1.17-revert to move directories back cluster-wise |
[production] |
| 13:38 |
<RoanKattouw> |
Moving directories back on /home |
[production] |
| 13:38 |
<RoanKattouw> |
Rolling back 1.17 deployment |
[production] |
| 13:31 |
<catrope> |
synchronized php-1.5/includes/parser/Parser.php 'r81738, fixes registerWatcher fatal' |
[production] |
| 13:30 |
<catrope> |
synchronized php-1.5/extensions/NewUserMessage/NewUserMessage.class.php 'r81737, fixes fatal' |
[production] |
| 13:08 |
<catrope> |
synchronizing Wikimedia installation... Revision: 81732: |
[production] |
| 13:07 |
<RoanKattouw> |
Running scap to make sure the move takes effect properly, caches are purged, etc. |
[production] |
| 13:07 |
<RoanKattouw> |
Moving wmf-deployment to w-d.old and php-1.17 to wmf-deployment on all Apaches. Run 1.17-rollback to undo this |
[production] |
| 13:06 |
<RoanKattouw> |
Moving wmf-deployment to w-d.old and php-1.17 to wmf-deployment on /home |
[production] |
| 13:05 |
<catrope> |
ran sync-common-all |
[production] |
| 13:04 |
<RoanKattouw> |
Rerunning sync-common-all |
[production] |
| 13:03 |
<mark> |
Upgraded rsync to a version with higher tcp backlog on nfs2 |
[production] |
| 13:01 |
<RoanKattouw> |
sync-common-all failed for a few servers due to NFS problems. Mark is fixing it |
[production] |
| 12:58 |
<catrope> |
ran sync-common-all |
[production] |
| 12:57 |
<RoanKattouw> |
Moved old wmf-deployment/ and php-1.17/ back on /home , now syncing that out to the cluster |
[production] |
| 10:38 |
<mark> |
Power cycled amssq37, 43, 49 |
[production] |
| 10:34 |
<mark> |
Power cycled amssq35 |
[production] |
| 10:15 |
<mark> |
Deploying special case for test.wikipedia.org (=> srv193) on bits.pmtpa (varnish) |
[production] |
| 10:12 |
<catrope> |
synchronized live-1.5/load.php 'Revert failed hack' |
[production] |
| 10:01 |
<catrope> |
synchronized live-1.5/load.php 'Temp hack to make load.php work on test' |
[production] |
| 09:28 |
<root> |
ran sync-common-all |
[production] |
| 08:58 |
<root> |
ran sync-common-all |
[production] |
| 08:41 |
<mark> |
Unstuck SMS modem |
[production] |
| 07:50 |
<Tim> |
restarted apache on srv174 |
[production] |
| 07:22 |
<mark> |
Started Nagios bot |
[production] |
| 06:15 |
<^demon> |
fixed svn permissions for wmf-deployment group |
[production] |
|
2011-02-07
§
|
| 22:20 |
<Ryan_Lane> |
upgrading nova-* on nova-*.tesla |
[production] |
| 20:02 |
<jeluf> |
synchronized php-1.5/wmf-config/InitialiseSettings.php '27195 - Grant anonymous users 'createpage' right and enable Extension' |
[production] |
| 19:55 |
<jeluf> |
synchronized php-1.5/wmf-config/InitialiseSettings.php '27209 - Request Extension of Quiz Addon to English Wikibooks' |
[production] |
| 17:18 |
<demon> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Removing proof-of-concept MWReleases extension' |
[production] |
| 17:18 |
<demon> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'Disabling proof-of-concept MWReleases extension' |
[production] |
| 13:18 |
<Tim> |
running schema updates on all old masters. db29 already done. |
[production] |
| 13:17 |
<Tim> |
new master positions: db22-bin.000001:106, db18-bin.000001:106, db34-bin.000001:106 |
[production] |
| 13:16 |
<tstarling> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 13:16 |
<tstarling> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 13:15 |
<Tim> |
doing master switch on s3 from db39 to db34 |
[production] |
| 13:15 |
<tstarling> |
synchronized php-1.5/wmf-config/db.php |
[production] |