5801-5850 of 10000 results (24ms)
2010-03-08 §
13:20 <RoanKattouw> Rebuilding all l10n caches to prep debugging a LocalisationUpdate issue [production]
13:16 <domas> something was wrong with sq31/sq32 - cpu/network shot through roof, can't correlate load [production]
10:42 <domas> *sigh*, squid conf: added highPrecedence to defaults, so it is picked up by configuration builder. also, checked in files into RCS (had to take over locks from mr.Starling, sorry!). unbroke test.wikipedia API [production]
09:43 <domas> where did test api,php fix go?! [production]
04:38 <Fred> removed Bart's ACL in squid cache conf since it was a duplicate ACL. [production]
2010-03-06 §
22:59 <mark> Increased membufs on all backend squids [production]
06:58 <aaron> synchronized php-1.5/wmf-config/InitialiseSettings.php 'disabled FlaggedRevs_alpha due to outdated messages issues' [production]
05:40 <aaron> synchronized php-1.5/wmf-config/flaggedrevs.php '$wgFlaggedRevsStylePath for labs' [production]
05:35 <aaron> synchronized php-1.5/wmf-config/flaggedrevs.php [production]
05:33 <aaron> synchronized php-1.5/wmf-config/flaggedrevs.php 'Set labs site to use flaggedrevs_alpha' [production]
05:26 <aaron> synchronized php-1.5/wmf-config/InitialiseSettings.php 'Set labs site to use flaggedrevs_alpha' [production]
05:10 <aaron> ran sync-common-all [production]
00:18 <root> synchronized php-1.5/wmf-config/InitialiseSettings.php 'Bug 22728 - create new user group on Greek Wiktionary' [production]
2010-03-05 §
18:48 <Fred> removed duplicate ACL from squid backend config. [production]
18:11 <Rob> restarted npre daemon on ms1 since it wasnt transmitting to nagios. [production]
00:51 <aaron> ran sync-common-all [production]
00:23 <mark> synchronized php-1.5/wmf-config/InitialiseSettings.php 'Restore Monobook for nl.wikimedia.org' [production]
2010-03-04 §
19:44 <Tim> installed NRPE and RAID utils on ms1 [production]
19:40 <Tim> removed db19 from the mysql node group and nagios since it is being used for varnish now [production]
19:29 <Tim> fixed prototype.wikimedia.org (nagios critical), was waiting 10s on every request due to memcached being down [production]
2010-03-03 §
16:16 <apergos> reenables gmetric on ms8 a little bit ago, scrub finished with no errors (yeah right( [production]
06:06 <apergos> replication off again tonight, zfs nicely borked on ms8, zfs destroy snapshot X gives message "cannot destroy X, dataset already exists" . time for sun contract and call them [production]
05:53 <apergos> gmetrics turned off again on ms8, running zpool scrub export [production]
05:40 <apergos> reset SYs on ms8 worked (slowly). gmetrics reenabled. (and I thought morebots was just being a bad bot) [production]
05:39 <apergos> ( :-P ) starting catchup for the last day's lost replication ms7 -> ms8 [production]
05:22 <apergos> reboot failed, bootadm hung probably waiting on zfs. stop SYS claims to work but subsequent show SYS shows power state on. [production]
05:14 <apergos> rebootiing ms8 to clear zfs deadlock [production]
2010-03-02 §
23:32 <catrope> synchronized php-1.5/extensions/UsabilityInitiative/OptIn/SpecialOptIn.php 'r63191' [production]
23:31 <catrope> synchronized php-1.5/extensions/UsabilityInitiative/OptIn/OptIn.php 'r63191' [production]
23:31 <catrope> synchronized php-1.5/wmf-config/CommonSettings.php 'Tweak OptIn settings' [production]
22:36 <RoanKattouw> Rebuilding localisation cache [production]
20:15 <Tim> on hume: started recompressTracked.php for all wikis, to fix any newly-tracked pointers from fixBug20757.php and to complete enwiki [production]
03:48 <atglenn> leaving replication and gmetrics turned off on ms8 for thenight, will go back to looking at scat and mdb output tomorrow. [production]
02:27 <atglenn> turning off gmetric on ms8, the dfs are piling up behind a deadlocked zfs list [production]
02:13 <atglenn> turn of ms7 replication to ms8 temporarily to work out a zfs deadlock issue on ms8 [production]
2010-03-01 §
12:53 <kate> synchronized php-1.5/wmf-config/db.php [production]
12:51 <river> removing db17 from rotation for ts dump [production]
12:51 <kate> synchronized php-1.5/wmf-config/db.php [production]
11:48 <domas> recovered locke, ran log collection with per-process limit of 200MB [production]
11:39 <domas> locke was killed by awks using gigabytes of memory, mwahaha. [production]
2010-02-27 §
02:09 <Tim> rewrote domas's api.php fix and deployed my version [production]
01:56 <midom> synchronized php-1.5/includes/LocalisationCache.php 'removing profiling hooks' [production]
01:42 <domas> fixed api.php on test/srv124 [production]
2010-02-26 §
23:40 <Tim> disabled the temporary squid purge script, I'm assuming it's done its job now [production]
19:43 <catrope> synchronized php-1.5/includes/api/ApiBase.php 'r63014, r63015' [production]
18:24 <Fred> bugzilla now residing on Kaulen... Updating server roles. [production]
18:07 <Fred> switching bugzilla cname to new server. (both servers are pointing to the same DB, so switch should be transparent to most) [production]
17:28 <midom> synchronized php-1.5/includes/LocalisationCache.php 'profiling' [production]
17:26 <midom> synchronized php-1.5/includes/diff/DifferenceInterface.php 'removing profiling hook' [production]
17:23 <midom> synchronized php-1.5/includes/StubObject.php 'tracehooks' [production]