6451-6500 of 10000 results (30ms)
2009-11-26 §
13:18 <mark> Running apt-get dist-upgrade on pdf1, in part to upgrade libpoppler (PDF rendering library) [production]
13:18 <mark> Removed rogue apt repositories on pdfd1 [production]
02:24 <Tim> started apache on spence [production]
2009-11-25 §
23:46 <tfinc> synchronized php-1.5/wmf-config/CommonSettings.php [production]
18:16 <Fred> rebooted wikitech linode / added to Nagios for basic monitoring. [production]
16:28 <mark> Starting NFS failover test on nfs1 & nfs2 [production]
14:57 <mark> Setup syslog-ng server on nfs1/nfs2 with puppet. Flipped syslog-ng.pmtpa.wmnet CNAME to point to the NFS service ip [production]
2009-11-24 §
23:50 <fvassard> synchronized php-1.5/wmf-config/CommonSettings.php 'Adding 2009_Notice30_bold' [production]
23:00 <midom> synchronized php-1.5/wmf-config/db.php 'deebee seventeen and twenteeseven' [production]
22:13 <midom> synchronized php-1.5/wmf-config/db.php 'adding db21 and db29 back' [production]
21:20 <midom> synchronized php-1.5/wmf-config/db.php 'taking out db17 and db29 too' [production]
21:16 <midom> synchronized php-1.5/wmf-config/db.php 'taking out db21 and db27, AND NO PUPPETS' [production]
21:14 <mark> Setup logrotate for /home/w/logs/ on both nfs1/nfs2 (of course, with puppet ;-) [production]
19:07 <mark> Resized JFS filesystem of /home on nfs2 to fit the new LV size on nfs1/nfs2 [production]
19:03 <mark> Moved cronned rsync of /home from db20 to nfs1 and nfs2, managed by puppet [production]
18:46 <mark> MediaWiki logging, broken by yesterday's NFS service IP migration, restored (using puppet) [production]
18:15 <mark> Puppetised rsyncd setup on nfs1/nfs2 [production]
18:00 <andrew> synchronized php-1.5/wmf-config/InitialiseSettings.php 'Flood flag for strategywiki bug 20347' [production]
01:43 <Tim> enabled write-back caching on db14 despite broken controller battery, due to excessive lag and MASTER_POS_WAIT() [production]
00:24 <fvassard> synchronized php-1.5/wmf-config/CommonSettings.php 'It turns out that this time FiveFacts is actually referenced as Five_Facts_About_Wikipedia... who knew...' [production]
00:10 <fvassard> synchronized php-1.5/wmf-config/CommonSettings.php 'It turns out that this time FiveFacts is actually referenced as Five_Facts_About_Wikipedia... who knew...' [production]
00:01 <fvassard> synchronized php-1.5/wmf-config/CommonSettings.php 'Adding Notice_[36,37,28] and FiveFacts landing page' [production]
2009-11-23 §
21:42 <fvassard> synchronized php-1.5/wmf-config/CommonSettings.php 'Adding RU and IL for GEOIP location.' [production]
21:27 <fvassard> synchronized php-1.5/wmf-config/CommonSettings.php 'Adding RU and IL for GEOIP location.' [production]
20:53 <river> moving ptolemy to TS vlan to reinstall it as the TS OSM db [production]
20:11 <mark> Finished migrating NFS off db20 to a DRBD cluster of nfs2 (primary) and nfs1 (secondary) [production]
19:14 <RoanKattouw> prototype got stuck, rebooted it. Linode sucks and we should not use it ever again [production]
16:53 <mark> Re-signed puppet cert for fenari... why was it gone? [production]
12:39 <domas> spence was taken down by runaway rrdtool process, http://p.defau.lt/?8FAf9FtfQTbkSQKCcP_qmw [production]
11:00 <Andrew> Scapping to prod message updates [production]
10:49 <andrew> synchronized php-1.5/extensions/LiquidThreads_alpha/pages/NewUserMessagesView.php 'Merge r59353' [production]
10:35 <Andrew> Scapping [production]
10:34 <Andrew> Updating LiquidThreads to trunk state on alpha only [production]
08:28 <domas> sq14 has failed sdc [production]
02:05 <Tim> fixed clear-profile so that it works on fenari and other ubuntu hosts [production]
01:54 <tstarling> synchronized php-1.5/index.php 'test' [production]
01:54 <Tim> set up fenari's /root/.ssh/config like the one on zwinger, with its user known hosts file pointing to the global one [production]
01:41 <Tim> Moved logmsgbot to fenari. Opened up its UDP port to zwinger and added firewall rules to /etc/rc.local for honour-system access control. [production]
01:21 <Tim> commented out the nagios-specific nickserv stuff in ircecho, so that logmsgbot doesn't ghost nagios-wm [production]
01:11 <Tim> installed udprec at fenari:/usr/local/bin/udprec [production]
00:55 <Tim> on fenari, symlinked ddsh -> dsh (dancer's) so that the sync scripts can work [production]
2009-11-22 §
14:15 <domas> powercycled locke after 10h downtime [production]
10:45 <RoanKattouw> Reports that file deletion and undeletion on Commons is broken [production]
03:35 <rainman-sr> rebuilding mwsuggest index for en.wp which throws strange lucene exceptions, hopefully fresh built is going to solve it [production]
01:24 <rainman-sr> deployed latest lucene-search to test liquidthreads integration [production]
2009-11-21 §
21:03 <domas> sq13 needs reboot and probably hardware maintenance [production]
21:03 <domas> sq43 and sq37 need hdd replacements. [production]
20:46 <domas> squid peermonitor is receiving 504s from peers... [production]
20:45 <domas> firewalled too. [production]
20:44 <domas> cleaned up sq13 crap in remote logs on db20, was full / fs [production]