|
2011-08-11
§
|
| 17:57 |
<mark> |
Temporarily moved squids->apaches LVS traffic from lvs4 to lvs3 for testing |
[production] |
| 16:48 |
<Ryan_Lane> |
upping the nginx upload size too 100m for the https and ipv6 cluster |
[production] |
| 16:06 |
<Reedy> |
SVN and related services should be ok now. High HTTP load causing OOM |
[production] |
| 15:51 |
<Reedy> |
SVN may be unavailable due to issues with Formey |
[production] |
| 15:32 |
<catrope> |
synchronized php/includes/filerepo/LocalFile.php '[[rev:94252|r94252]] by Chad - Try wrapping ss_images update in a transaction' |
[production] |
| 15:17 |
<RoanKattouw> |
All file uploads were returning HTTP 500 errors between 15:07 and 15:16 UTC, my apologies. It's fixed now |
[production] |
| 15:16 |
<catrope> |
synchronized wmf-config/StartProfiler.php 'Fix it for real this time' |
[production] |
| 15:15 |
<catrope> |
synchronized wmf-config/StartProfiler.php 'Unbreak uploads, oops' |
[production] |
| 15:07 |
<catrope> |
synchronized wmf-config/StartProfiler.php 'Make upload profiling 1:1 instead of 1:50' |
[production] |
| 15:01 |
<mark> |
Set maximum TCP window size on nas1-a |
[production] |
| 15:00 |
<RoanKattouw> |
...and that worked. Lesson of the day: it seems you can't use dashes in your profile IDs |
[production] |
| 14:59 |
<catrope> |
synchronized wmf-config/StartProfiler.php 'Remove dashes from profile IDs just because I'm paranoid' |
[production] |
| 14:56 |
<catrope> |
synchronized wmf-config/StartProfiler.php 'Add profiling groups upload-commons and upload-other, based on count($_FILES)' |
[production] |
| 14:10 |
<mark> |
Setup semi-sync snapmirror replication from nas1001-a:images to nas1-a:images, started initial transfer |
[production] |
| 13:46 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Bug 8886 - Install DynamicPageList extension for Vietnamese Wiktionary' |
[production] |
| 13:37 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php '[[bugzilla:18390|bug 18390]] Set [bureaucrat][] = sysop on enwiki' |
[production] |
| 13:34 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php '[[bugzilla:30307|bug 30307]] Allow bureaucrats to remove sysop flag on Russian Wikipedia' |
[production] |
| 12:45 |
<mark> |
Migrated nas1-a and nas1-b to new 64 bit root volumes |
[production] |
| 09:45 |
<catrope> |
synchronized wmf-config/checkers.php 'Remove broken profiling in wfACLBlocks. Not needed because everything in $wgExtensionFunctions is automatically profiled anyway' |
[production] |
| 09:17 |
<mutante> |
sq75 - power cycle, squid clean |
[production] |
| 03:00 |
<LocalisationUpdate> |
completed (1.17) at Thu Aug 11 03:02:09 UTC 2011 |
[production] |
| 02:32 |
<LocalisationUpdate> |
completed (1.17) at Thu Aug 11 02:34:20 UTC 2011 |
[production] |
| 02:31 |
<Reedy> |
LocalisationUpdate is giving Warning: array_diff_assoc(): Argument #1 is not an array in /home/wikipedia/common/php-1.17/extensions/LocalisationUpdate/LocalisationUpdate.class.php on line 373 |
[production] |
| 01:58 |
<LocalisationUpdate> |
completed (1.17) at Thu Aug 11 02:00:53 UTC 2011 |
[production] |
| 01:56 |
<LocalisationUpdate> |
completed (1.17) at Thu Aug 11 01:58:21 UTC 2011 |
[production] |
| 01:46 |
<LocalisationUpdate> |
completed at Thu Aug 11 01:48:34 UTC 2011 |
[production] |
| 01:46 |
<LocalisationUpdate> |
failed |
[production] |
| 01:30 |
<LocalisationUpdate> |
failed |
[production] |
| 01:27 |
<LocalisationUpdate> |
failed |
[production] |
| 01:12 |
<LocalisationUpdate> |
failed |
[production] |
| 00:42 |
<demon> |
synchronized wmf-config/StartProfiler.php 'Enabled profiling group for commonswiki' |
[production] |
|
2011-08-10
§
|
| 23:02 |
<LocalisationUpdate> |
failed |
[production] |
| 22:59 |
<LocalisationUpdate> |
failed |
[production] |
| 21:25 |
<preilly> |
pushing device change back into production |
[production] |
| 21:25 |
<preilly> |
synchronizing Wikimedia installation... Revision: 93960: |
[production] |
| 20:49 |
<reedy> |
synchronized wmf-config/CommonSettings.php 'Revert my change' |
[production] |
| 20:49 |
<preilly> |
fixing mobile view |
[production] |
| 20:49 |
<reedy> |
synchronized wmf-config/CommonSettings.php 'Temporarily disabling MObileFrontend' |
[production] |
| 20:49 |
<preilly> |
synchronizing Wikimedia installation... Revision: 93960: |
[production] |
| 20:48 |
<RoanKattouw> |
MobileFrontend broke, serving mobile pages to everyone. Fixing |
[production] |
| 20:43 |
<preilly> |
pushing fix for Mobile Frontend device detection |
[production] |
| 20:43 |
<preilly> |
synchronizing Wikimedia installation... Revision: 93960: |
[production] |
| 19:02 |
<preilly> |
synchronizing Wikimedia installation... Revision: 93960: |
[production] |
| 19:02 |
<preilly> |
updating mobile frontend extension on production |
[production] |
| 18:41 |
<notpeter> |
upgrading squid and squid-frontend on amssq49 |
[production] |
| 17:01 |
<mark> |
Deployed new HTCP loss ganglia monitoring plugin on ganglia aggregator squids in pmtpa |
[production] |
| 16:32 |
<mutante> |
sq51 & sq62 - dist-upgrade (incl. kernel) / reboot |
[production] |
| 16:01 |
<mutante> |
sq51,sq62: power cycle, squid clean |
[production] |
| 02:18 |
<LocalisationUpdate> |
completed at Wed Aug 10 02:21:00 UTC 2011 |
[production] |
| 00:12 |
<maplebed> |
trying out a change to puppet's nagios config generation - moving service checks to one file per host. |
[production] |