|
2013-12-09
§
|
| 09:36 |
<apergos> |
restart backend varnish on cp4019, out of memory (fork failed and other nice things) |
[production] |
| 09:25 |
<apergos> |
restarted opendj on sanger |
[production] |
| 09:12 |
<apergos> |
powercycled sq80, i/o errors for every command, fs complaints on various filesystems in dmesg, smells like controller but it booted ok so... |
[production] |
| 09:01 |
<apergos> |
powercycled ms-be1006, unresponsive, lots of "BUG: soft lockup - CPU#0 stuck for 22s!" etc from time it stopped responding (16.00 utc Sunday) |
[production] |
| 04:38 |
<springle> |
synchronized wmf-config/db-eqiad.php 'repool es1010 after upgrade, max_connections lowered during warm up' |
[production] |
| 03:21 |
<springle> |
synchronized wmf-config/db-eqiad.php 'depool es1010 for upgrade' |
[production] |
| 03:12 |
<springle> |
synchronized wmf-config/db-eqiad.php 'repool es1007 after upgrade, max_connections lowered during warm up' |
[production] |
| 02:49 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Mon Dec 9 02:49:20 UTC 2013 |
[production] |
| 02:43 |
<springle> |
synchronized wmf-config/db-eqiad.php 'depool es1007 for upgrade' |
[production] |
| 02:21 |
<LocalisationUpdate> |
completed (1.23wmf6) at Mon Dec 9 02:21:01 UTC 2013 |
[production] |
| 02:11 |
<LocalisationUpdate> |
completed (1.23wmf5) at Mon Dec 9 02:11:34 UTC 2013 |
[production] |
| 02:00 |
<springle> |
resync labsdb dewiki.pagelinks from upstream |
[production] |
|
2013-12-06
§
|
| 23:12 |
<K4-713> |
un-updated payments cluster back to 671411e77f18a9b... again |
[production] |
| 23:10 |
<K4-713> |
updated payments to fb96b8bfb1695cd725cdd |
[production] |
| 22:45 |
<mutante> |
mediawiki 1.22 has been released - http://dumps.wikimedia.org/mediawiki/1.22/ |
[production] |
| 21:44 |
<demon> |
synchronized wmf-config/InitialiseSettings.php 'Fix undefined variable' |
[production] |
| 20:59 |
<reedy> |
synchronized docroot/bits/favicon/office.ico 'Ib5332726fd02874b25213dd7813b59a684197ac3' |
[production] |
| 20:56 |
<ori-l> |
Restarting gmetad on nickel |
[production] |
| 20:24 |
<K4-713> |
un-updated payments cluster back to 671411e77f18a9b |
[production] |
| 20:18 |
<K4-713> |
updated payments cluster to 9312d81ef0186edfc |
[production] |
| 20:14 |
<ori-l> |
Restarting gmetad on nickel for Ib111e53b6 |
[production] |
| 20:06 |
<ori> |
synchronized wmf-config/throttle.php 'Ifbd6417a2: add throttle exception for queens public lib event' |
[production] |
| 20:05 |
<ori> |
updated /a/common to {{Gerrit|Ifbd6417a2}}: add throttle exception for queens public lib event |
[production] |
| 19:28 |
<yurik> |
synchronized wmf-config/flaggedrevs.php |
[production] |
| 18:55 |
<mholmquist> |
synchronized php-1.23wmf6/extensions/UploadWizard/resources/mw.UploadWizardDetails.js 'Fix UploadWizard for IE8' |
[production] |
| 18:54 |
<mholmquist> |
synchronized php-1.23wmf5/extensions/UploadWizard/resources/mw.UploadWizardDetails.js 'Fix UploadWizard for IE8' |
[production] |
| 17:37 |
<reedy> |
synchronized php-1.23wmf6/extensions/Wikibase 'I7cf7af5525f3223dbd044e7676b5c0255b45928c' |
[production] |
| 17:37 |
<reedy> |
synchronized php-1.23wmf6/extensions/DataValues 'I7cf7af5525f3223dbd044e7676b5c0255b45928c' |
[production] |
| 17:34 |
<reedy> |
synchronized php-1.23wmf5/extensions/Wikibase 'I06994427b780cd0b66dc0b0279045df7699aef1c' |
[production] |
| 10:28 |
<hashar> |
synchronized php-1.23wmf6/extensions/ProofreadPage 'ProofreadPage bug fixes {{gerrit|99627}}' |
[production] |
| 10:16 |
<hashar> |
synchronized php-1.23wmf5/extensions/ProofreadPage 'ProofreadPage bug fixes {{gerrit|99522}}' |
[production] |
| 10:02 |
<apergos> |
shot and restarted gmond processes on hooft |
[production] |
| 04:27 |
<springle> |
synchronized wmf-config/db-eqiad.php 'repool es1001 after upgrade, max_connections lowered during warm up' |
[production] |
| 03:33 |
<reedy> |
synchronized multiversion/MWMultiVersion.php |
[production] |
| 02:51 |
<springle> |
set es1004 max_connections back to default (should have been reset previously, hence dberror log noise) |
[production] |
| 02:46 |
<springle> |
synchronized wmf-config/db-eqiad.php 'depool es1001 for upgrade' |
[production] |
| 02:29 |
<springle> |
synchronized wmf-config/db-eqiad.php 'direct vslow query group to LB=0 snapshot slaves' |
[production] |
| 02:27 |
<tstarling> |
synchronized wmf-config/trusted-xff.cdb |
[production] |