|
2008-12-15
§
|
| 22:22 |
<RobH> |
changed settings on metawiki to allow banned users to edit their talk pages per https://bugzilla.wikimedia.org/show_bug.cgi?id=16621 |
[production] |
| 21:25 |
<brion> |
reenabling handheld skin setting, was turned off during overload emergencies on 11-17 |
[production] |
| 21:13 |
<brion> |
rsyncd appears to be running on srv56. does anything else need to be done for index updates? |
[production] |
| 20:10 |
<brion> |
yongle hanging again, restarting apache |
[production] |
| 18:58 |
<RobH> |
started rsync daemon on srv56 per rainman |
[production] |
| 18:35 |
<RobH> |
setup new planet per https://bugzilla.wikimedia.org/show_bug.cgi?id=16511. |
[production] |
| 01:39 |
<brion-weekend> |
applying API deletion log fix from r44541 ([[bugzilla:16626]]) |
[production] |
| 00:09 |
<rainman-sr> |
rsyncd is not running on srv56, updates for wikis served by old indexer halted since Oct7. Run rsync --daemon on srv56 |
[production] |
|
2008-12-12
§
|
| 22:47 |
<brion> |
patched Bugzilla so we can exclude CC-only mails from wikibugs-l ([bugzilla:15585]]) |
[production] |
| 21:52 |
<brion> |
scapping to r44509 |
[production] |
| 19:19 |
<brion> |
put all the themes and plugins and patches back on wordpress for blog.wm.o. whee |
[production] |
| 19:15 |
<brion> |
restarted apache on isidore while fiddling with php error logging settings and blog started magically working again. sigh. going back to tweak its config back to normal |
[production] |
| 18:04 |
<brion> |
we managed to fix the svn update conflict on blog.wm.o (to wordpress 2.7) but it's still showing main page as blank |
[production] |
| 17:42 |
<mark> |
Telia connection / BGP session was up for 20 hours; problem seems resolved. Removed route filters |
[production] |
| 00:29 |
<brion> |
bumping to r44485 for more NS fixes for ms, ast |
[production] |
| 00:12 |
<brion> |
scapping bump to r44484, fixing a few issues w/ hu |
[production] |
| 00:06 |
<brion> |
updated wikibugs irc script to r44483, fixes issues w/ users w/o real name setting |
[production] |
|
2008-12-11
§
|
| 23:19 |
<brion> |
shutting down srv118; bad config. missing upload5 mount, seems to have bogus authenticatin (local su to root fails with "Authentication service cannot retrieve authentication info") |
[production] |
| 23:10 |
<brion> |
restarted apache on 134, it's scary/corrupt |
[production] |
| 22:55 |
<brion> |
manually syncing updated skin files to upload.wm.o ... |
[production] |
| 22:53 |
<brion> |
scapping to r44474 |
[production] |
| 21:31 |
<brion> |
don't sync yet; RC regression in r44033 being worked on |
[production] |
| 19:41 |
<brion-codereview> |
removed conflicting live profiling hack from AutoLoader.php. Put this stuff in SVN, huh guys? |
[production] |
| 19:39 |
<brion-codereview> |
applying flaggedrevs schema updates |
[production] |
| 19:38 |
<brion-codereview> |
starting svn up for testwiki |
[production] |
| 13:41 |
<mark> |
configured asw-a4-sdtpa and asw-a5-sdtpa, but no link |
[production] |
| 10:41 |
<mark> |
bart out of disk space, removed some old cruft (mailman) |
[production] |
|
2008-12-10
§
|
| 23:50 |
<RobH> |
pulled srv76 due to two dead fans (yay for da bot) |
[production] |
| 23:35 |
<RobH> |
srv78 reinstalled and in apache pool |
[production] |
| 22:57 |
<RobH> |
srv78 kernel panic, old FC install, pulled for reinstall |
[production] |
| 22:49 |
<RobH> |
sq1. sq3, sq6 cache cleaned and back online serving requests. |
[production] |
| 22:35 |
<RobH> |
sq1, sq3, sq6 all unresponsive to console, flashing leds on kvm. rebooted. |
[production] |
| 20:40 |
<RobH> |
srv118 installation completed. |
[production] |
| 20:00 |
<RobH> |
reinstalled srv118 after replacing dead parts. installing packages now. |
[production] |
| 19:48 |
<RobH> |
started rebuild of storage1 /c1/p0 into array |
[production] |
| 19:47 |
<RobH> |
replaced disk /c1/p0 in storage1. /c1/p13 is now bad as well, placing rma for it. |
[production] |
| 19:14 |
<RobH> |
db13-db16 responsive to ssh. |
[production] |
| 19:13 |
<RobH> |
db15 rebooted. |
[production] |
| 18:05 |
<RobH> |
temp probes installed in a3-sdtpa |
[production] |