|
2012-01-26
§
|
| 02:37 |
<RoanKattouw> |
Started the udp2log process for the AFT logger manually on emery |
[production] |
| 02:06 |
<LocalisationUpdate> |
completed (1.18) at Thu Jan 26 02:06:22 UTC 2012 |
[production] |
| 01:01 |
<asher> |
synchronized wmf-config/db.php 'adding db32 to s1 at low weight, new enwiki snapshot host' |
[production] |
| 00:33 |
<asher> |
synchronized wmf-config/db.php 'returning db18, now replicating heartbeat db' |
[production] |
| 00:28 |
<asher> |
synchronized wmf-config/db.php 'temporarily pulling db18' |
[production] |
| 00:11 |
<pgehres> |
re-enabling recurring donation module and processing in CiviCRM |
[production] |
| 00:04 |
<asher> |
synchronized wmf-config/db.php 'adding db26 to s7' |
[production] |
| 00:03 |
<RobH> |
shutting down srv151-srv186 per RT 2318 (confirmed not in pybal pools for apache or api) |
[production] |
|
2012-01-25
§
|
| 23:59 |
<binasher> |
removed old external store apaches from pybal config |
[production] |
| 23:50 |
<asher> |
synchronized wmf-config/db.php 'pulling db26' |
[production] |
| 23:49 |
<asher> |
synchronized wmf-config/db.php 'adding db26 to s7' |
[production] |
| 23:35 |
<pgehres> |
re-enabled queue consumption for payments through Jenkins |
[production] |
| 23:35 |
<pgehres> |
awjr synchronized CiviCRM on aluminium to [[rev:1211|r1211]] |
[production] |
| 23:05 |
<awjrichards> |
synchronized wmf-config/CommonSettings.php 'Checking if wmgDisplayFeedsInSidebar === false rather than true, since it defaults to true in the install file' |
[production] |
| 22:50 |
<awjrichards> |
synchronizing Wikimedia installation... : Enabling FeaturedFeeds everywhere |
[production] |
| 22:41 |
<RobH> |
updating dns for bellin/blondel db9/10 replacements |
[production] |
| 22:32 |
<awjrichards> |
synchronizing Wikimedia installation... : Dark-deploying FeaturedFeeds |
[production] |
| 22:07 |
<awjrichards> |
synchronized wmf-config/CommonSettings.php 'fixed spelling mistake fore FeaturedFeeds configuration' |
[production] |
| 22:05 |
<awjrichards> |
synchronized wmf-config/CommonSettings.php 'Setting up FeaturedFeeds config; disabled by default' |
[production] |
| 22:02 |
<awjrichards> |
synchronized wmf-config/InitialiseSettings.php 'Setting up FeaturedFeeds config; disabled by default' |
[production] |
| 21:54 |
<LeslieCarr> |
deactivated selected-paths policy-statement on cr1-eqiad and cr2-eqiad |
[production] |
| 21:23 |
<Tim> |
on srv197: compiled and installed a local version of wmerrors for segfault investigation |
[production] |
| 21:22 |
<binasher> |
bits caches: running varnish param.set thread_pool_min, thread_pool_max, where min = 15000 / cores / 4 and max = 15000 / cores |
[production] |
| 20:57 |
<binasher> |
running "varnishadm param.set thread_pool_max 1875" on mobile varnish servers |
[production] |
| 20:05 |
<Tim> |
on srv197: temporarily disabled puppet and enabled core dumps in apache2.conf for segfault flood investigation |
[production] |
| 19:57 |
<Jeff_Green> |
running dist-upgrade on payments* and silicon |
[production] |
| 19:44 |
<Tim> |
updating TrustedXFF host list using fenari |
[production] |
| 19:29 |
<RobH> |
dns update go! |
[production] |
| 18:50 |
<LeslieCarr> |
restarted varnish on niobium |
[production] |
| 18:49 |
<RoanKattouw> |
Restarted morebots |
[production] |
| 13:33 |
<demon> |
synchronized wmf-config/CommonSettings.php 'Change IP address for bnwiki account creation throttle per [[bugzilla:33900|bug 33900]]' |
[production] |
| 11:43 |
<apergos> |
formey oom (I guess), unresponsive from mgmt console, powercycling. |
[production] |
| 06:07 |
<pgehres> |
disabled queue consumption of payments in jenkins until stuck message can be removed from queue |
[production] |
| 05:49 |
<pgehres> |
Disabling the processing of recurring payments in CiviCRM until we can add the appropriate payment_method to the queue msgs |
[production] |
| 02:06 |
<LocalisationUpdate> |
completed (1.18) at Wed Jan 25 02:06:17 UTC 2012 |
[production] |
| 00:57 |
<binasher> |
streaming hotbackup of db53 to db32 |
[production] |
| 00:52 |
<binasher> |
shutting down mysql on db32, going to reconfigure with lvm and reslave |
[production] |
| 00:45 |
<asher> |
synchronized wmf-config/db.php 'pulling db32 - this will be the new enwiki pmtpa snapshot host' |
[production] |
| 00:30 |
<binasher> |
streaming hotbackup of db37 to db26, preparing to reprovision db26 in s7 |
[production] |
|
2012-01-24
§
|
| 23:30 |
<pgehres> |
testing |
[production] |
| 23:28 |
<awjr> |
Syncing prod CiviCRM on aluminium to [[rev:1209|r1209]] |
[production] |
| 22:39 |
<mark> |
Disabled quota support on sanger's IMAP server to make Dovecot work again |
[production] |
| 22:23 |
<mark> |
Sanger is upgraded to lucid |
[production] |
| 21:49 |
<RobH> |
ms-be1 is online! MAN WE ARE AWESOME |
[production] |
| 21:20 |
<mark> |
Starting dist-upgrade of sanger |
[production] |
| 20:50 |
<binasher> |
pulled db26, rebooting and re-imaging with lucid |
[production] |
| 20:48 |
<asher> |
synchronized wmf-config/db.php 'pulling db26 from s1 to reimage' |
[production] |
| 20:31 |
<cmjohnson1> |
restarting ms4 for memory testing |
[production] |
| 20:19 |
<notpeter> |
spinning up db54-58 for asher |
[production] |
| 20:09 |
<asher> |
synchronized wmf-config/db.php 're-weighting s6 dbs' |
[production] |