351-400 of 709 results (5ms)
2008-12-10 §
22:35 <RobH> sq1, sq3, sq6 all unresponsive to console, flashing leds on kvm. rebooted. [production]
20:40 <RobH> srv118 installation completed. [production]
20:00 <RobH> reinstalled srv118 after replacing dead parts. installing packages now. [production]
19:48 <RobH> started rebuild of storage1 /c1/p0 into array [production]
19:47 <RobH> replaced disk /c1/p0 in storage1. /c1/p13 is now bad as well, placing rma for it. [production]
19:14 <RobH> db13-db16 responsive to ssh. [production]
19:13 <RobH> db15 rebooted. [production]
18:05 <RobH> temp probes installed in a3-sdtpa [production]
2008-12-09 §
18:46 <RobH> fixed group names in add/remove groups per https://bugzilla.wikimedia.org/show_bug.cgi?id=16248 [production]
18:42 <RobH> updated some settings for no.wikimedia.org and pushed to cluster. [production]
15:23 <RobH> backedup blog frontend/database and upgraded to 2.6.5 successfully [production]
14:21 <RobH> updated InitialiseSettings for nowikimedia wiki [production]
02:03 <brion> dropped loading of obsolete RenderHash ext (bug 16114) [production]
2008-12-08 §
23:30 <RobH> updated enwiktionary group settings per https://bugzilla.wikimedia.org/show_bug.cgi?id=16248 [production]
23:24 <brion> updating Oversight for bug 16065 [production]
22:44 <RobH> no.wikimedia.org is now functioning per https://bugzilla.wikimedia.org/show_bug.cgi?id=15383 [production]
22:35 <RobH> made changes to InitialiseSettings.php for cswikisource per https://bugzilla.wikimedia.org/show_bug.cgi?id=16277 [production]
21:38 <RobH> authdns-update for no.wikimedia.org [production]
21:20 <RobH> running sync-common-all for wikimedia norge (found the php error) [production]
21:01 <RobH> its all back up now. [production]
20:59 <RobH> I stupidly crashed the site with a php typo, rolling back my changes since i was ignorant and did not php -l ;_; [production]
20:58 <RobH> setup wikimedia norge wiki per https://bugzilla.wikimedia.org/show_bug.cgi?id=15383 [production]
19:23 <brion> updating OggHandler for fix for bug 15920 (chopped oggs) [production]
15:57 <mark> Set up mirroring of traffic of e7/2 to e7/14 for testing the fiber patch loop/optics [production]
2008-12-05 §
22:42 <domas> srv47 is running scaler usr.sbin.apache2 aa profile in learning mode [production]
22:33 <RobH> sq50 reinstalled and back in rotation [production]
22:26 <RobH> finished setup on srv146, back in apache pool [production]
21:32 <RobH> setting up packages on srv146 [production]
21:32 <RobH> reinstalling sq50 [production]
21:27 <brion> pointing SiteMatrix at local copy, not NFS master, of langlist file [production]
19:19 <RobH> added sq48, and sq49 back into pool. sq50 pending reinstallation. [production]
18:58 <mark> depooled broken squids sq1 and sq3 [production]
18:26 <RobH> depooled sq48-sq50 for relocation [production]
18:17 <RobH> added sq44-sq47 back into pybal, relocation complete. [production]
17:45 <brion> sync-common-all to add w/test-headers.php [production]
17:28 <RobH> shutting down sq44-sq47 for relocation. [production]
17:27 <RobH> sq41 - sq43 back online. [production]
17:17 <RobH> sq40 oddness, but its back up now [production]
16:44 <RobH> accidentally pulled power for sq38, opps! [production]
15:36 <RobH> removed sq41 - sq43 from pybal to relocate from pmtpa to sdtpa [production]
15:34 <domas> srv178 running usr.sbin.apache2 aa profile in complain mode [production]
15:34 <RobH> removed sq40 from pybal to relocate from pmtpa to sdtpa [production]
2008-12-04 §
22:50 <domas> job runners are no longer blue on ganglia CPU graphs :((((((( [production]
22:45 <domas> fc4 maintenance, reniced job runners to 20 (10 behind apaches), installed apc3.0.19 (APC3.0.13 seams to have hit severe lock contention/busylooping at overloads) [production]
22:04 <RobH> re-enabled sq38 in pybal. all is well [production]
22:02 <RobH> fired sq37-sq39 back up [production]
21:58 <RobH> shutdown sq37-sq39, cuz I need to balance the power distribution a bit better. [production]
21:40 <RobH> sq38 is trying to break my spirit, so i reinstalled it to show it who is boss (me!) [production]
21:02 <RobH> setup asw-a4-sdtpa and asw-a5-sdtpa on scs-a1-sdtpa [production]
20:52 <mark> Increased TCP buffers on srv88 (a Fedora), matching the Ubuntus - Fedora Apaches appear to get stuck/deadlocked on writes to Squids [production]