|
2010-02-10
§
|
| 20:46 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'Deploy r62262' |
[production] |
| 19:22 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' |
[production] |
| 19:21 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'Deploying r62257' |
[production] |
| 19:05 |
<Rob> |
storage1 disks replaced, except for the newly dead one, rma placed. |
[production] |
| 18:39 |
<Rob> |
shutting down storage1 to replace its bad OS mirror disk |
[production] |
| 16:52 |
<mark> |
Fixed MySQL permissions on srv186 as well |
[production] |
| 16:15 |
<mark> |
Rigged puppet to deploy a PHP mail.ini file to the apaches, which sets it to call sendmail with -f <> (sending mail with empty envelope sender, like bounces) |
[production] |
|
2010-02-09
§
|
| 22:06 |
<Andrew> |
LiquidThreads production software updates completed successfully. |
[production] |
| 21:58 |
<andrew> |
synchronized php-1.5/extensions/LiquidThreads/classes/Dispatch.php |
[production] |
| 21:57 |
<mark> |
Firewalled one ip on singer which was overloading Apache, looked like a Wordpress attack |
[production] |
| 21:50 |
<mark> |
Lowered MaxClients setting to 250 on singer |
[production] |
| 21:47 |
<Andrew> |
Running scap to deploy LiquidThreads updates |
[production] |
| 21:45 |
<Andrew> |
Updating LiquidThreads production deployments to the alpha version in the next few minutes. |
[production] |
| 21:44 |
<mark> |
Power cycled singer |
[production] |
| 21:19 |
<mark> |
Fixed MySQL system user & permissions on srv151-srv185 |
[production] |
| 21:17 |
<mark> |
Fixed MySQL instance on srv183 |
[production] |
| 20:47 |
<mark> |
Fixed MySQL instance on srv156 |
[production] |
| 19:39 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' |
[production] |
| 19:29 |
<catrope> |
synchronized php-1.5/skins/common/edit.js 'Deploy r62190. Only for test for now, thank God for style versions' |
[production] |
| 17:01 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' |
[production] |
| 17:01 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'Deploying r62187' |
[production] |
| 16:52 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' |
[production] |
| 16:51 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'Deploying r62184' |
[production] |
| 14:19 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' |
[production] |
| 14:19 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'Deploying r62181' |
[production] |
| 02:14 |
<andrew> |
synchronized php-1.5/extensions/LiquidThreads_alpha/lqt.js 'Merge r62158' |
[production] |
|
2010-02-08
§
|
| 23:25 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix for r62041. Varnish needs this or Tampa will serve stale JS off bits' |
[production] |
| 23:06 |
<mark> |
Deploying Exim on all application servers as well |
[production] |
| 22:06 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/OptIn/SpecialOptIn.php 'Deploy r62141' |
[production] |
| 21:51 |
<mark> |
Added search3 and search9 to search_pool_1 on lvs3 |
[production] |
| 21:48 |
<mark> |
Deploying Exim as a simple mail sending-only MTA (with queuing enabled) on squids, image scalers, search servers and core databases |
[production] |
| 21:38 |
<Andrew> |
LiquidThreads wikinews rollout complete, though there was some buggage related to replication that had to be resolved. |
[production] |
| 21:29 |
<andrew> |
synchronized php-1.5/extensions/LiquidThreads_alpha/classes/Dispatch.php |
[production] |
| 21:29 |
<andrew> |
synchronized php-1.5/extensions/LiquidThreads_alpha/classes/Dispatch.php |
[production] |
| 21:27 |
<andrew> |
synchronized php-1.5/extensions/LiquidThreads_alpha/classes/Dispatch.php |
[production] |
| 21:24 |
<andrew> |
synchronized php-1.5/extensions/LiquidThreads_alpha/classes/Dispatch.php |
[production] |
| 21:22 |
<andrew> |
synchronized php-1.5/extensions/LiquidThreads_alpha/classes/Dispatch.php |
[production] |
| 21:21 |
<andrew> |
synchronized php-1.5/extensions/LiquidThreads_alpha/classes/Dispatch.php |
[production] |
| 21:18 |
<andrew> |
synchronized php-1.5/wmf-config/InitialiseSettings.php |
[production] |
| 21:18 |
<andrew> |
synchronized php-1.5/wmf-config/liquidthreads.php |
[production] |
| 21:13 |
<Andrew> |
Starting LiquidThreads deployment on enwikinews. |
[production] |
| 20:00 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Track NTOC and dialogs as well in PrefStats' |
[production] |
| 13:04 |
<Tim> |
running fixBug20757.php on enwiki, in a screen on hume |
[production] |
| 12:51 |
<Tim> |
locke went into swap half an hour ago and is now dead. Trying reboot. |
[production] |
| 07:38 |
<Tim> |
stopped recompressTracked.php for now due to bug 20757 |
[production] |
| 07:04 |
<tstarling> |
synchronized php-1.5/includes/Revision.php 'r62120' |
[production] |
| 06:18 |
<Tim> |
cleared out db9 root partition (which had 370MB free) by moving 4GB of SQL files from /root and /home/tfinc to /a/backup/junk |
[production] |
| 06:01 |
<tstarling> |
synchronized php-1.5/wmf-config/db.php 're-added ms2 as an ES slave for rc1,cluster22 and the ex-fedora clusters, was depooled for ~6 months' |
[production] |
| 05:55 |
<Tim> |
added grant for root@fenari to ms3/2 |
[production] |
| 05:31 |
<Tim> |
started apache on srv213, srv161 and srv240, was stopped for no apparent reason |
[production] |