51-100 of 2292 results (5ms)
2009-05-28 §
10:08 <tstarling> synchronized php-1.5/includes/specials/SpecialEmailuser.php [production]
07:40 <midom> synchronized php-1.5/db.php 'db26 going live' [production]
00:31 <rainman_> deployed udp search query logging on searchidx1 and search1-12 [production]
2009-05-27 §
20:37 <tomaszf> killing long running query. bugzilla is all well again [production]
20:33 <tomaszf> cleaning out numerous locks on db9 causing huge slowdown for bugzilla3 db [production]
08:48 <andrew> synchronized php-1.5/InitialiseSettings.php 'AbuseFilter on English Wikinews' [production]
2009-05-26 §
22:51 <river> deleted some old snapshots on ms1 [production]
22:45 <Fred> fixed dsh a tad, taking care of a few host which keys have been changed and hosts missing keys... [production]
22:28 <Fred> removed stale amane NFS mount on all apache boxes. [production]
22:17 <Andrew> added 'Change Tagging' component to bugzilla [production]
22:05 <brion> adminned werdna on bugzilla to help w/ component config etc [production]
18:17 <rainman-sr> all of search_4 wikis temporarely moved to search12 server, preparing search11 for taking its place with full set of lucene-search 2.1 features [production]
17:52 <robh> synchronized php-1.5/InitialiseSettings.php '18589 Enable AbuseFilter on Cantonese Wikipedia' [production]
17:51 <robh> synchronized php-1.5/abusefilter.php 'updates on bug 18589 Enable AbuseFilter on Cantonese Wikipedia' [production]
17:47 <robh> synchronized php-1.5/InitialiseSettings.php '18938 Please activate Rollback on Simple.Wikibooks' [production]
17:27 <andrew> synchronized php-1.5/lucene.php 'Rainman made me do it! (taking search11 out of rotation)' [production]
16:45 <rainman-sr> running initial warmup and start of lsearchd on search12 [production]
16:38 <Rob> search12 memory upgraded and rebooted [production]
16:37 <rainman-sr> running initial warmup and restart on search11 [production]
16:32 <Rob> search11 ram upgraded and rebooted. [production]
16:04 <Rob> srv217 shutdown until I can poke it [production]
16:01 <tstarling> synchronized php-1.5/CommonSettings.php 'removed "xx" from $wgStyleVersion' [production]
15:36 <Rob> decommissioning out of warranty and broken servers: srv53, srv67, srv85, srv88, srv90. [production]
15:22 <Rob> db28 fan controller board replaced (not db26, typo) system is now online [production]
15:21 <Rob> db26 fan controller board replaced, system is now online [production]
15:15 <Rob> db26 memory replaced, shows all 32 GB now. Restarting [production]
15:13 <andrew> synchronized php-1.5/includes/Export.php 'Removing useless free()s breaking partial dumps' [production]
14:20 <Rob> shutdown mysql on db26 as it will have the memory replaced in approx. 25 minutes. [production]
13:58 <tstarling> synchronized php-1.5/skins/common/block.js [production]
13:56 <Tim> merged r50871 [production]
13:55 <tstarling> synchronized php-1.5/includes/Block.php [production]
08:14 <Tim> srv78 had apache running but did not have /mnt/upload5 mounted. Upgraded its wikimedia-task-appserver package and mounted it. [production]
2009-05-25 §
21:35 <mark> Modified scap scripts to work on /home-less apaches [production]
21:17 <midom> synchronized php-1.5/db.php 'replaced db16 with db12 for auxiliary roles' [production]
18:51 <mark> Restarted stuck apache on srv187 [production]
18:49 <mark> db16 got in trouble at 18:35, spiking to 2000 threads. Might be related to the nv_nic_irq bug. It recovered 5 mins afterwards [production]
17:33 <mark> Increased COSS cache dirs from 10 GB to 15 GB on knsq16+ esams squids [production]
07:08 <domas> brought williams up, was down, console not responding. [production]
2009-05-23 §
01:07 <tomaszf> restarted srv159 due to hard down [production]
2009-05-22 §
18:03 <mark> Increased big object store on all upload squids from 15 GB to 20 GB [production]
15:24 <mark> Raised max in-memory object size to 100 kiB on all squids [production]
15:11 <mark> Raised cache dir size of large object store on knsq18 from 15 to 20 GB [production]
14:58 <mark> Increased cache dir size by 50% on knsq16, and upped max object size in memory from 75 to 100 kB followed by a backend squid restart [production]
14:40 <mark> Increased cache dir size by 50% on knsq16, and upped max object size in memory from 75 to 100 kB [production]
10:41 <mark> apt-get dist-upgrade on searchidx1 [production]
10:29 <mark> Rebooting searchidx1 [production]
09:40 <rainman-sr> zombie java process 1666 on searchidx1 locked in i/o and taking up lots of ram, cannot kill it, searchidx1 need restart [production]
2009-05-21 §
19:09 <Fred> restarted squid process on sq43 as it was not responding properly. [production]
14:08 <Tim> updated [[Switch master]] docs [production]
13:15 <domas> db12 RAID set to no-battery write-behind mode: <code>arcconf SETCACHE 1 LOGICALDRIVE 0 WB noprompt</code> [production]