101-150 of 230 results (3ms)
2008-11-12 §
18:46 <brion> added "uploader" group so we can bump known-good people into being able to upload without waiting for the autoconfirm heuristic [production]
2008-11-11 §
16:20 <RobH> search2 still down, drives will not detect reliably. Ticket with sun reopened. [production]
15:56 <RobH> replaced backplane on search2, reinstalling. [production]
15:13 <RobH> srv137 back online. apache and memcached back up. [production]
14:49 <RobH> srv100 back online. [production]
10:44 <river> removed centralnotice php from brion's crontab as it was breaking zwinger [production]
01:19 <brion> swapped Commons to use $wgNoticeProject 'wikimedia' rather than having separate 'Commons needs you' notices [production]
00:57 <brion> swapped in fundraiser to all projects [production]
2008-11-10 §
19:18 <mark> Shutdown AMS-IX route server 1 session as it's been flapping for hours [production]
2008-11-09 §
16:11 <river> removed nfsfind cronjob on ms1 [production]
2008-11-07 §
22:52 <brion_> tossing 2008_meter_2b notice into partial rotation on enwiki -- has reduced collapsed version [production]
22:50 <brion_> adding "_collapsed" to banner source tracking for collapsed view [production]
22:27 <brion> scapping updates to ContributionReporting and CentralNotice [production]
01:43 <Tim> experimentally reading the civicrm database into db10 with --master-data=1 [production]
01:19 <brion> db9 temporarily (hopefully) messed up. tim's fiddling with it to put it back [production]
00:09 <river> upgraded ms2 to solaris 10 update 6 [production]
2008-11-06 §
21:03 <Tim> switched GIFs to use Bitmap_ClientOnly (client-side scaling) [production]
17:23 <brion> restarting apache on srv47, seem smysteriously stuck [production]
17:15 <brion> setting $wgMaxAnimatedGifArea to 1 to prevent animated thumbnailing of GIFs for now, see if that helps [production]
17:10 <brion> river complaining of image scaler issues -- load spikes, depooling? [production]
01:42 <Tim> restarting squids [production]
01:42 <mark> Setup lvs4 as temp LVS support for lvs2, balancing the load [production]
01:07 <brion> updated ContributionReporting to add paging links to ContributionHistory (might be a little funky w/ caching, we'll work it out :) [production]
00:45 <Tim> progressively clearing /a on the remaining image scalers [production]
00:37 <Tim> wiping /a on srv44 [production]
00:00 <brion> scapping for update to ContributionReporting [production]
2008-11-05 §
23:38 <brion> set yongle to restart apache every hour since it still seems to bork up and get stuck sometimes [production]
22:01 <RobH> srv100 rebooted, was down. [production]
18:28 <mark> tech team is procrastinating [production]
18:16 <atglenn> added dhelps to office@wikimedia.org alias, redirected office@wikipedia.org to him also [production]
18:14 <brion> disabling centralnotice on private wikis, we don't need to be told to donate to ourselves ;) [production]
18:03 <brion> poking sitenotices off wikibooks, on *.wikipedia [production]
18:03 <brion> set up ariel on mchenry for mail admin [production]
05:38 <brion_> opera users may rejoice ;) [production]
05:38 <brion_> tweaked storage1 lighttpd config so centralnotice.js is served with utf-8 charset [production]
05:17 <brion_> for reference -- load spikes are page rendering on enwiki and dewiki mostly :) [production]
05:17 <brion_> bumping enwiki notice to 100% [production]
05:07 <Tim> killed various mysqld_safe processes which were using 100% CPU on ES servers [production]
04:50 <brion_> fixed morebots -- bots now allowed to edit again at wikitech [production]
04:50 <brion_> enabling enwiki notice at about 10% sampling [production]
03:27 <brion_> squids are... i think.... looking better :D [production]
02:47 <brion_> seems happier after restart of front-end squids [production]
02:43 <brion_> tim's doing hard restarts of more squids, we're kinda offline briefly [production]
02:34 <brion_> disabling centralnotices on remaining sites just for good measure while we debug [production]
02:29 <brion_> current status: the squids which borked are still kind of borked, but perhaps slightly better. mark is examining squid memory reports [production]
02:14 <brion> tim's attempting to restart borked squids [production]
02:01 <brion> disabling enwiki centralnotice while investigating hits dropoff [production]
2008-11-04 §
21:36 <Tim> added nagios monitoring of HTTP on image backends [production]
21:14 <Tim> installed NRPE stuff on db19 [production]
19:37 <Tim> killed the broken NFS mount on db21:/mnt with umount -l. The processes that are waiting for it will probably hang until system restart [production]