251-300 of 6085 results (4ms)
2010-02-19 §
18:48 <Fred> removed memcached node srv193 and replaced with srv233 [production]
16:52 <Rob> updated InitiliseSettings with bug 22585 [production]
01:31 <Fred> puppetized gmond.conf generation using templates [production]
2010-02-18 §
23:51 <RoanKattouw> Fixed search script on yongle, removed UA check exemption for yongle [production]
23:51 <RoanKattouw> logmsgbot is broken again [production]
04:07 <Tim> re-running storageTypeStats.php on enwiki to identify rows with old_flags='object,utf-8', which need to be handled properly in fixBug20757.php [production]
2010-02-17 §
17:25 <Rob> pushed updates to wordpress installations [production]
17:04 <RoanKattouw> Synced wmf-config/checkers.php 'Allow missing UA from yongle' [production]
17:04 <RoanKattouw> Restarted logmsgbot [production]
16:47 <Fred> test [production]
07:24 <Fred> there is a problem with ganglia. It is being worked on. [production]
00:15 <RoanKattouw> Resynced srv151, was returning empty responses [production]
2010-02-16 §
23:31 <Fred> upgrading gmond to 3.1.2 everywhere. However, due to the newish module structure, there is a potential that ganglia will hickup while puppet does its job... [production]
22:03 <logmsgbot_> mark synchronized php-1.5/wmf-config/checkers.php 'Exceptions' [production]
20:30 <Rob> srv127 is online, but not in LVS. Its cert was accepted on sockpuppet, but puppetd --test results in a cert failure on retrieval from sockpuppet. [production]
20:17 <Rob> rebooting srv127 [production]
20:15 <mark> Increased membufs to 40 per COSS dir on the pmtpa upload backend squids [production]
19:51 <mark> Increased membufs per COSS dir from 10 to 20 on the new pmtpa squids [production]
18:10 <apergos> but documentation can save people precious time when things are on fire [production]
18:02 <mark> Documentation is not a substitution for thinking [production]
16:44 <mark> Fixed puppet on most servers [production]
16:23 <domas> anyone knows why mysqldump on snapshot3 is locking tables? maybe --single-transaction could work better?!!!? [production]
16:22 <RoanKattouw> Strike my last, I hear it'll fix itself in a day [production]
16:21 <RoanKattouw> Oops, meant to type srv187-189 [production]
16:21 <RoanKattouw> Ganglia not picking up data for srv1987-189, 193, 214-218, 250-253, 257 even though the boxes are up and gmond is running; been like this for 3 days [production]
15:21 <mark> Removing all puppet certs/private keys on all machines [production]
14:55 <mark> Puppetmaster is screwed up by a wrong command that deleted all files under /var/lib/puppet [production]
14:25 <logmsgbot_> midom synchronized php-1.5/includes/diff/DifferenceInterface.php 'instrumenting costs by revision pairs' [production]
13:46 <logmsgbot_> root synchronized php-1.5/wmf-config/InitialiseSettings.php 'bug 22444 and 22330 enabling collection on dawiki and ruwiki' [production]
13:22 <domas> oh wait, it does work, something else.... [production]
13:16 <domas> apparently my squid rule for Apple browsers didn't really work, hence the continuing spikes [production]
03:16 <domas> blocked MacOSX Atom syndication on the edge, this will save terabytes of diskspace on unsuspecting Safari user computers :-) [production]
02:13 <logmsgbot_> tstarling synchronized php-1.5/wmf-config/checkers.php [production]
2010-02-15 §
23:42 <logmsgbot_> midom synchronized php-1.5/includes/StubObject.php 'uninstrumenting' [production]
23:09 <logmsgbot_> midom synchronized php-1.5/wmf-config/checkers.php [production]
23:09 <domas> banned all UA-less requests [production]
22:53 <logmsgbot_> midom synchronized php-1.5/wmf-config/checkers.php 'un-disabling UA check on API' [production]
22:14 <RoanKattouw> Resynced srv114, was returning empty responses [production]
22:10 <logmsgbot_> mark synchronized php-1.5/wmf-config/CommonSettings.php 'Test with 220px default thumb size on enwiki (bug #21117)' [production]
22:10 <logmsgbot_> mark synchronized php-1.5/wmf-config/InitialiseSettings.php 'Test with 220px default thumb size on enwiki (bug #21117)' [production]
22:02 <logmsgbot_> catrope synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' [production]
21:58 <logmsgbot_> catrope synchronized php-1.5/extensions/UsabilityInitiative/Vector/Vector.combined.min.js 'r62553' [production]
21:39 <logmsgbot_> catrope synchronized php-1.5/includes/api/ApiMain.php 'r62550' [production]
21:30 <logmsgbot_> mark synchronized php-1.5/wmf-config/InitialiseSettings.php 'raised wgSearchSuggestCacheExpiry from 1200 to 86400' [production]
21:27 <logmsgbot_> catrope synchronized php-1.5/includes/api/ApiOpenSearch.php 'r62548' [production]
21:27 <logmsgbot_> catrope synchronized php-1.5/includes/api/ApiMain.php 'r62548' [production]
21:27 <logmsgbot_> catrope synchronized php-1.5/includes/DefaultSettings.php 'r62548' [production]
20:54 <logmsgbot_> catrope synchronized php-1.5/includes/api/ApiBase.php 'r62546' [production]
20:54 <logmsgbot_> catrope synchronized php-1.5/api.php 'r62546' [production]
20:53 <domas> Roan and me are doing some perf engineering on opensearch & related stuff. \\o/ [production]