751-800 of 1152 results (6ms)
2008-12-16 §
20:29 <RobH> ariel is fubar, rebooting and investigating. [production]
20:25 <RobH> restarted services on sq13 [production]
20:21 <RobH> took down sq13 to clean its cache [production]
20:09 <RobH> replaced bad /c0/p0 in amane [production]
18:47 <brion> applying temporary resource limit lift on enwiki for an IP for workshop in SF [production]
17:40 <RobH> updated dns for misc. servers project. [production]
01:08 <brion> deploying r44643 update to CodeReview subversion proxy (swapped encoding protocol to avoid bugs in json_decode with some diffs) [production]
00:04 <brion> running cleanupTitles.php in bg on all wikis... [production]
2008-12-15 §
23:20 <brion> going to test fixes for FiveUpgrade.inc to back cleanupTitles.php, cleanupImages.php etc [production]
22:22 <RobH> changed settings on metawiki to allow banned users to edit their talk pages per https://bugzilla.wikimedia.org/show_bug.cgi?id=16621 [production]
21:25 <brion> reenabling handheld skin setting, was turned off during overload emergencies on 11-17 [production]
21:13 <brion> rsyncd appears to be running on srv56. does anything else need to be done for index updates? [production]
20:10 <brion> yongle hanging again, restarting apache [production]
18:58 <RobH> started rsync daemon on srv56 per rainman [production]
18:35 <RobH> setup new planet per https://bugzilla.wikimedia.org/show_bug.cgi?id=16511. [production]
01:39 <brion-weekend> applying API deletion log fix from r44541 ([[bugzilla:16626]]) [production]
00:09 <rainman-sr> rsyncd is not running on srv56, updates for wikis served by old indexer halted since Oct7. Run rsync --daemon on srv56 [production]
2008-12-14 §
02:04 <Platonides> Connections timing out [production]
2008-12-13 §
02:04 <brion> applied patch-rfb_ratings.sql to flaggedrevs wikis [production]
01:46 <brion> did some debugging on RatingHistory graph generation with Aaron and got it working yay! [production]
2008-12-12 §
22:47 <brion> patched Bugzilla so we can exclude CC-only mails from wikibugs-l ([bugzilla:15585]]) [production]
21:52 <brion> scapping to r44509 [production]
19:19 <brion> put all the themes and plugins and patches back on wordpress for blog.wm.o. whee [production]
19:15 <brion> restarted apache on isidore while fiddling with php error logging settings and blog started magically working again. sigh. going back to tweak its config back to normal [production]
18:04 <brion> we managed to fix the svn update conflict on blog.wm.o (to wordpress 2.7) but it's still showing main page as blank [production]
17:42 <mark> Telia connection / BGP session was up for 20 hours; problem seems resolved. Removed route filters [production]
00:29 <brion> bumping to r44485 for more NS fixes for ms, ast [production]
00:12 <brion> scapping bump to r44484, fixing a few issues w/ hu [production]
00:06 <brion> updated wikibugs irc script to r44483, fixes issues w/ users w/o real name setting [production]
2008-12-11 §
23:19 <brion> shutting down srv118; bad config. missing upload5 mount, seems to have bogus authenticatin (local su to root fails with "Authentication service cannot retrieve authentication info") [production]
23:10 <brion> restarted apache on 134, it's scary/corrupt [production]
22:55 <brion> manually syncing updated skin files to upload.wm.o ... [production]
22:53 <brion> scapping to r44474 [production]
21:31 <brion> don't sync yet; RC regression in r44033 being worked on [production]
19:41 <brion-codereview> removed conflicting live profiling hack from AutoLoader.php. Put this stuff in SVN, huh guys? [production]
19:39 <brion-codereview> applying flaggedrevs schema updates [production]
19:38 <brion-codereview> starting svn up for testwiki [production]
13:41 <mark> configured asw-a4-sdtpa and asw-a5-sdtpa, but no link [production]
10:41 <mark> bart out of disk space, removed some old cruft (mailman) [production]
2008-12-10 §
23:50 <RobH> pulled srv76 due to two dead fans (yay for da bot) [production]
23:35 <RobH> srv78 reinstalled and in apache pool [production]
22:57 <RobH> srv78 kernel panic, old FC install, pulled for reinstall [production]
22:49 <RobH> sq1. sq3, sq6 cache cleaned and back online serving requests. [production]
22:35 <RobH> sq1, sq3, sq6 all unresponsive to console, flashing leds on kvm. rebooted. [production]
20:40 <RobH> srv118 installation completed. [production]
20:00 <RobH> reinstalled srv118 after replacing dead parts. installing packages now. [production]
19:48 <RobH> started rebuild of storage1 /c1/p0 into array [production]
19:47 <RobH> replaced disk /c1/p0 in storage1. /c1/p13 is now bad as well, placing rma for it. [production]
19:14 <RobH> db13-db16 responsive to ssh. [production]
19:13 <RobH> db15 rebooted. [production]