|
2009-12-10
§
|
| 04:55 |
<atglenn> |
unpacking the last three months of incrementals on ms7 |
[production] |
| 02:19 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 00:34 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'stress testing hehehehehe' |
[production] |
| 00:27 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 00:26 |
<tstarling> |
synchronized php-1.5/wmf-config/db.php 'increasing db12 load' |
[production] |
| 00:24 |
<Andrew> |
db12 recovered, db26 still barely coping with the load |
[production] |
| 00:17 |
<tstarling> |
synchronized php-1.5/wmf-config/db.php 'unloading general load from db12, excessive lag >800s' |
[production] |
| 00:09 |
<Andrew> |
db12 seems not to be replicating at all. |
[production] |
| 00:09 |
<Andrew> |
make that db12, not db22 |
[production] |
| 00:06 |
<Andrew> |
s1 lagged, db12 282s, db26 33s -- no slave servers up-to-date enough to serve pages |
[production] |
|
2009-12-09
§
|
| 23:53 |
<fred> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Adding 2009_Notice_43-44-45 to tracking.' |
[production] |
| 23:06 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'taking down one more server oh noes' |
[production] |
| 22:59 |
<Fred> |
fixed noc.wikimedia.org |
[production] |
| 21:08 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'db14 offline' |
[production] |
| 20:54 |
<atglenn> |
open support call with sun for ms8, won't power up (no POST), even after resetting ILOM |
[production] |
| 17:28 |
<andrew> |
ran sync-common-all |
[production] |
| 17:18 |
<Andrew> |
Updating production LiquidThreads installations to new beta version, branch of current alpha. |
[production] |
| 17:08 |
<andrew> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'Bug 20253 Install Extension:NewUserMessage on strategy wiki' |
[production] |
| 13:39 |
<andrew> |
synchronized php-1.5/extensions/LiquidThreads_alpha/lqt.js 'Merge r59886' |
[production] |
| 13:38 |
<andrew> |
synchronized php-1.5/extensions/LiquidThreads_alpha/classes/View.php 'Merge r59885' |
[production] |
| 13:08 |
<andrew> |
synchronized php-1.5/extensions/LiquidThreads_alpha/classes/View.php 'Merge r59883' |
[production] |
| 13:05 |
<andrew> |
synchronized php-1.5/extensions/LiquidThreads_alpha/jquery/js2.combined.js 'merge r59882' |
[production] |
| 12:50 |
<andrew> |
ran sync-common-all |
[production] |
| 12:39 |
<Andrew> |
Updating LiquidThreads to trunk state, using sync-common-all to deploy to apaches (since scap crashes the cluster) |
[production] |
| 00:04 |
<RoanKattouw> |
Manually reallocating edits from [[User:Until It Sleeps]] to [[User:The Thing That Should Not Be]] on enwiki |
[production] |
|
2009-12-08
§
|
| 23:51 |
<catrope> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'bug 21718: Allow sysops to add/remove abusefilter too on arwiki' |
[production] |
| 23:46 |
<Big_Fred> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Added tracking for Notice_42' |
[production] |
| 23:31 |
<catrope> |
synchronized php-1.5/includes/DefaultSettings.php 'Deploying r59858' |
[production] |
| 23:31 |
<catrope> |
synchronized php-1.5/includes/Skin.php 'Deploying r59858' |
[production] |
| 23:31 |
<catrope> |
synchronized php-1.5/skins/vector/main-rtl.css 'Deploying r59858' |
[production] |
| 23:31 |
<catrope> |
synchronized php-1.5/skins/vector/main-ltr.css 'Deploying r59858' |
[production] |
| 23:31 |
<catrope> |
synchronized php-1.5/skins/common/ajaxwatch.js 'Deploying r59858' |
[production] |
| 23:26 |
<TrevorParscal> |
Running sync-file repeatedly to deploy r59858 (usability fixes) |
[production] |
| 23:16 |
<catrope> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'bug 21718: allow bureaucrats to add&remove abusefilter group on arwiki' |
[production] |
| 23:12 |
<catrope> |
synchronized php-1.5/wmf-config/abusefilter.php 'bug 21718: Restrict abusefilter-modify to abusefilter group on arwiki' |
[production] |
| 22:13 |
<tfinc> |
synchronized php-1.5/extensions/CentralNotice/SpecialNoticeText.php 'picking up r59852' |
[production] |
| 19:09 |
<Big_Fred> |
synchronized php-1.5/wmf-config/CommonSettings.php 'added geo-location for Italie.' |
[production] |
| 16:43 |
<andrew> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'Bug 20216 AbuseFilter for itwikiquote' |
[production] |
| 16:43 |
<andrew> |
synchronized php-1.5/wmf-config/abusefilter.php 'Bug 20216 AbuseFilter for itwikiquote' |
[production] |
| 16:20 |
<Rob> |
running diagnostics on srv217, please do not try to put it into service. |
[production] |
| 16:06 |
<Rob> |
srv229 memory replaced, system restarted. |
[production] |
| 15:58 |
<Rob> |
shutting down srv229 to swap out faulty dimm |
[production] |
| 15:46 |
<Rob> |
sq43 disk replaced, system back online and in rotation |
[production] |
| 15:31 |
<Rob> |
replaced bad disk in sq43 |
[production] |
| 15:23 |
<Rob> |
replaced bad disk in sq37, detected, restarted squid backend |
[production] |
| 15:18 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 15:09 |
<Rob> |
db27 back up with mysql running. |
[production] |
| 15:07 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 15:03 |
<Rob> |
mysql stopped normally on db27, restarting it. |
[production] |
| 15:02 |
<Rob> |
was able to hotswap the fan, shutdown was not needed |
[production] |