|
2012-01-26
§
|
| 22:14 |
<RobH> |
poking at puppet change breaking things on sockpuppet puppet runs |
[production] |
| 21:33 |
<^demon> |
gallium: cleared a bunch of junk from /tmp |
[production] |
| 21:12 |
<Jeff_Green> |
upgraded storage3 mysqld from 5.1.47 to mysql-at-facebook-[[rev:3753|r3753]] |
[production] |
| 20:17 |
<asher> |
synchronized wmf-config/db.php 'db37 back in s7' |
[production] |
| 20:10 |
<Reedy> |
Created "spoofuser" AntiSpoof table in the central auth database |
[production] |
| 19:34 |
<asher> |
synchronized wmf-config/db.php 'pulling db37 from s7 for upgrades' |
[production] |
| 19:24 |
<RobH> |
disregard any flapping by mw1001, its my script testbed |
[production] |
| 18:04 |
<RobH> |
forcing puppet run on srv199 |
[production] |
| 17:45 |
<RobH> |
shutting down srv199 for bios tinkering by chris |
[production] |
| 02:37 |
<RoanKattouw> |
Started the udp2log process for the AFT logger manually on emery |
[production] |
| 02:06 |
<LocalisationUpdate> |
completed (1.18) at Thu Jan 26 02:06:22 UTC 2012 |
[production] |
| 01:01 |
<asher> |
synchronized wmf-config/db.php 'adding db32 to s1 at low weight, new enwiki snapshot host' |
[production] |
| 00:33 |
<asher> |
synchronized wmf-config/db.php 'returning db18, now replicating heartbeat db' |
[production] |
| 00:28 |
<asher> |
synchronized wmf-config/db.php 'temporarily pulling db18' |
[production] |
| 00:11 |
<pgehres> |
re-enabling recurring donation module and processing in CiviCRM |
[production] |
| 00:04 |
<asher> |
synchronized wmf-config/db.php 'adding db26 to s7' |
[production] |
| 00:03 |
<RobH> |
shutting down srv151-srv186 per RT 2318 (confirmed not in pybal pools for apache or api) |
[production] |
|
2012-01-25
§
|
| 23:59 |
<binasher> |
removed old external store apaches from pybal config |
[production] |
| 23:50 |
<asher> |
synchronized wmf-config/db.php 'pulling db26' |
[production] |
| 23:49 |
<asher> |
synchronized wmf-config/db.php 'adding db26 to s7' |
[production] |
| 23:35 |
<pgehres> |
re-enabled queue consumption for payments through Jenkins |
[production] |
| 23:35 |
<pgehres> |
awjr synchronized CiviCRM on aluminium to [[rev:1211|r1211]] |
[production] |
| 23:05 |
<awjrichards> |
synchronized wmf-config/CommonSettings.php 'Checking if wmgDisplayFeedsInSidebar === false rather than true, since it defaults to true in the install file' |
[production] |
| 22:50 |
<awjrichards> |
synchronizing Wikimedia installation... : Enabling FeaturedFeeds everywhere |
[production] |
| 22:41 |
<RobH> |
updating dns for bellin/blondel db9/10 replacements |
[production] |
| 22:32 |
<awjrichards> |
synchronizing Wikimedia installation... : Dark-deploying FeaturedFeeds |
[production] |
| 22:07 |
<awjrichards> |
synchronized wmf-config/CommonSettings.php 'fixed spelling mistake fore FeaturedFeeds configuration' |
[production] |
| 22:05 |
<awjrichards> |
synchronized wmf-config/CommonSettings.php 'Setting up FeaturedFeeds config; disabled by default' |
[production] |
| 22:02 |
<awjrichards> |
synchronized wmf-config/InitialiseSettings.php 'Setting up FeaturedFeeds config; disabled by default' |
[production] |
| 21:54 |
<LeslieCarr> |
deactivated selected-paths policy-statement on cr1-eqiad and cr2-eqiad |
[production] |
| 21:23 |
<Tim> |
on srv197: compiled and installed a local version of wmerrors for segfault investigation |
[production] |
| 21:22 |
<binasher> |
bits caches: running varnish param.set thread_pool_min, thread_pool_max, where min = 15000 / cores / 4 and max = 15000 / cores |
[production] |
| 20:57 |
<binasher> |
running "varnishadm param.set thread_pool_max 1875" on mobile varnish servers |
[production] |
| 20:05 |
<Tim> |
on srv197: temporarily disabled puppet and enabled core dumps in apache2.conf for segfault flood investigation |
[production] |
| 19:57 |
<Jeff_Green> |
running dist-upgrade on payments* and silicon |
[production] |
| 19:44 |
<Tim> |
updating TrustedXFF host list using fenari |
[production] |
| 19:29 |
<RobH> |
dns update go! |
[production] |
| 18:50 |
<LeslieCarr> |
restarted varnish on niobium |
[production] |
| 18:49 |
<RoanKattouw> |
Restarted morebots |
[production] |
| 13:33 |
<demon> |
synchronized wmf-config/CommonSettings.php 'Change IP address for bnwiki account creation throttle per [[bugzilla:33900|bug 33900]]' |
[production] |
| 11:43 |
<apergos> |
formey oom (I guess), unresponsive from mgmt console, powercycling. |
[production] |
| 06:07 |
<pgehres> |
disabled queue consumption of payments in jenkins until stuck message can be removed from queue |
[production] |
| 05:49 |
<pgehres> |
Disabling the processing of recurring payments in CiviCRM until we can add the appropriate payment_method to the queue msgs |
[production] |
| 02:06 |
<LocalisationUpdate> |
completed (1.18) at Wed Jan 25 02:06:17 UTC 2012 |
[production] |
| 00:57 |
<binasher> |
streaming hotbackup of db53 to db32 |
[production] |
| 00:52 |
<binasher> |
shutting down mysql on db32, going to reconfigure with lvm and reslave |
[production] |
| 00:45 |
<asher> |
synchronized wmf-config/db.php 'pulling db32 - this will be the new enwiki pmtpa snapshot host' |
[production] |
| 00:30 |
<binasher> |
streaming hotbackup of db37 to db26, preparing to reprovision db26 in s7 |
[production] |