501-550 of 6403 results (4ms)
2010-03-02 §
03:48 <atglenn> leaving replication and gmetrics turned off on ms8 for thenight, will go back to looking at scat and mdb output tomorrow. [production]
02:27 <atglenn> turning off gmetric on ms8, the dfs are piling up behind a deadlocked zfs list [production]
02:13 <atglenn> turn of ms7 replication to ms8 temporarily to work out a zfs deadlock issue on ms8 [production]
2010-03-01 §
12:53 <kate> synchronized php-1.5/wmf-config/db.php [production]
12:51 <river> removing db17 from rotation for ts dump [production]
12:51 <kate> synchronized php-1.5/wmf-config/db.php [production]
11:48 <domas> recovered locke, ran log collection with per-process limit of 200MB [production]
11:39 <domas> locke was killed by awks using gigabytes of memory, mwahaha. [production]
2010-02-27 §
02:09 <Tim> rewrote domas's api.php fix and deployed my version [production]
01:56 <midom> synchronized php-1.5/includes/LocalisationCache.php 'removing profiling hooks' [production]
01:42 <domas> fixed api.php on test/srv124 [production]
2010-02-26 §
23:40 <Tim> disabled the temporary squid purge script, I'm assuming it's done its job now [production]
19:43 <catrope> synchronized php-1.5/includes/api/ApiBase.php 'r63014, r63015' [production]
18:24 <Fred> bugzilla now residing on Kaulen... Updating server roles. [production]
18:07 <Fred> switching bugzilla cname to new server. (both servers are pointing to the same DB, so switch should be transparent to most) [production]
17:28 <midom> synchronized php-1.5/includes/LocalisationCache.php 'profiling' [production]
17:26 <midom> synchronized php-1.5/includes/diff/DifferenceInterface.php 'removing profiling hook' [production]
17:23 <midom> synchronized php-1.5/includes/StubObject.php 'tracehooks' [production]
17:21 <midom> synchronized php-1.5/includes/StubObject.php 'tracehooks' [production]
14:38 <RoanKattouw> logmsgbot is missing in action AGAIN, starting it as me on fenari [production]
14:37 <RoanKattouw> Synced r63008 [production]
12:17 <domas> *sigh*, recovered locke after OOM, added few entries to limits.conf [production]
2010-02-25 §
23:48 <Tim> running the new fixBug20757.php on all wikis [production]
21:48 <RoanKattouw> Removed empty line from the end of all.dblist, broke account creation ^^ [production]
20:18 <RoanKattouw> Removed those closed wikis again, db lists back to normal [production]
20:08 <RoanKattouw> Temp adding some closed wikis to all.dblist to be able to dump them (again) [production]
19:01 <Rob> removed old servers will, bart, etc from DNS [production]
18:32 <Rob> pushing more dns changes for LOM on new servers. [production]
00:59 <Tim> made the squid purge script on locke also purge redirects from /w/index.php?, for bug 22639 [production]
00:32 <Tim> installed ack-grep on fenari [production]
2010-02-24 §
20:48 <Rob> IP conflicts on alsted/ersch/formey/kaulen. I must have typo'd something in one of their DRAC configurations because they are all messed up. will go into DC and physically review settings via console. [production]
16:10 <Rob> had formey and alsted ip info mixed up in dns, updating dns [production]
13:13 <RoanKattouw> Adding optin_survey table to wikis that didn't have it [production]
01:38 <atglenn> ganglia 3.1 packages up on ms7 and ms8 (see atg for solaris packages for other boxen) [production]
01:20 <domas> that was fun, let's do it again! [production]
01:17 <mark> Implemented udp2log filter on locke that pipes redirected URLs to a modified version of purgeList to clean the squid caches [production]
00:59 <Rob> added austin to the mortals user category for shell access on puppet, updating singer's account list [production]
00:30 <mark> Fixing wikipedia.be vhost ServerAlias setting in redirects.conf [production]
00:14 <domas> redefined tiertwo acl in squid to allow fenari/zwinger purges, purged special:random, restarted mobile1 stuff [production]
2010-02-23 §
23:41 <domas> stopped nginx on en.m until we resolve special:random issue [production]
20:53 <root> synchronized php-1.5/wmf-config/InitialiseSettings.php 'Bug 22628 - Logo of Chamorro Wikipedia' [production]
17:23 <Rob> drac setup on new servers ersch, formey, alsted, and kaulen. still need port allocation. [production]
16:47 <Rob> forgot to add mgmt, updating dns [production]
16:43 <Rob> updating dns for new servers ersch, formey, alsted, & kaulen [production]
15:37 <Rob> FINALLY got in the last RMA drive for storage1, arrays were toast (losing 1/2 the drives in a raid10 and having the unlucky chance of them being paired), reinstall in progress [production]
12:50 <RoanKattouw> Cleaned up my dblist voodoo, everything should be back to normal now [production]
12:41 <RoanKattouw> Fixed sync-dblist, was broken [production]
12:31 <RoanKattouw> Syncing pmtpa.dblist anyway, seems I have to [production]
12:24 <RoanKattouw> Temporarily adding wikis listed in bug 20325 to pmtpa.dblist so I can export their contents. Please don't sync this [production]
2010-02-22 §
23:40 <Tim> restarted replication on ms2 [production]