51-100 of 2998 results (4ms)
2009-07-20 §
18:09 <hcatlin> restarted mobile1 cluster to load in new software [production]
15:55 <Fred> bounced apache on srv193 [production]
07:18 <Tim> re-enabled CentralNotice [production]
07:18 <tstarling> synchronized php-1.5/wmf-config/InitialiseSettings.php [production]
07:13 <apergos> enough data removed from ms1 to feel safe for a few days; started mass copy of remaining thumbs to ms4 in prep for complete repo switchover (running in root screen on ms1) [production]
04:48 <Tim> copying up all available MW release files from my laptop [production]
04:21 <Tim> mounted ms4:/export/dumps on zwinger [production]
04:16 <Tim> changed export options for ms4:/export/dumps to allow root access for the local subnet [production]
01:19 <hcatlin> On mobile1 we are now gzipping the log files after rotation in /srv/wikimedia-mobile/logs [production]
2009-07-19 §
22:02 <Fred> restarted memcached on srv159 [production]
20:43 <mark> eiximenis backend squid pooled [production]
20:10 <mark> Restarted deadlocked powerdns on bayle [production]
19:14 <mark> Installed eiximenis with a Squid OS install [production]
18:58 <mark> Moved eiximenis to vlan 100 (squids) [production]
18:55 <mark> Changed eiximenis' IP into 208.80.152.119 for Squid testing [production]
17:41 <hcatlin> Mobile1's web stack just got switched from Phusion Passenger to Nginx/Thin/Rack. [production]
2009-07-18 §
15:23 <apergos> some thumb directories on ms4 created at request of img scalers were created with owner root and perms 700... fixing [production]
03:55 <river> ms5 is ready [production]
01:20 <atglenn> continuing with removals of thumbs on ms1. 789G free now, need to reach about 1450 before we can just "maintain". but we're gaining on it. [production]
00:22 <brion> set up temporary data dump index, copied the dvd index (it's just offsite links). still need to track some MW releases [production]
00:07 <brion> recovering MediaWiki 1.6 through 1.10 release files and re-uploading them... [production]
2009-07-17 §
23:42 <brion> added a 404 page and recovered index.php for our temp download.wikimedia.org [production]
22:05 <brion> set wikitech to use vector skin by default :D [production]
22:03 <Andrew> Fixed morebots, which was relying on a fragile version check. Just deleted it :) [production]
22:02 <Andrew> testing [production]
20:43 <brion> fixed paths for noc.wikimedia.org/conf file highlighting [production]
20:38 <domas> ms2 has broken disks.. [production]
20:31 <brion> We're going to see about setting up the previously-idle ms5 so we can get our thumbnailing on [production]
20:01 <brion> rob's poking raid rebuild on storage2 (dumps server) [production]
19:03 <RobH_A90> eiximenis and dobson pulled for solid state drive testing, do not use for other tasks [production]
18:28 <brion> synchronized wmf-deployment/wmf-config/InitialiseSettings.php 'enabling vector for rtl' [production]
18:25 <atglenn> started mass move out of the way of thumbnail dirs and replacing with symlinks to ms4 [production]
18:25 <brion> synchronized wmf-deployment/wmf-config/CommonSettings.php 'bump style version' [production]
18:25 <brion> ran sync-common-all [production]
18:24 <brion> running sync-common-all for UI updates. need to poke the style ver too :) [production]
18:08 <brion> svn up'ing wmf-deployment for test.wikipedia.org. Merged UI fixes from usability team [production]
18:03 <Fred> spun a couple more apache server into image scalers: srv219..srv224. [production]
17:28 <rainman-sr> putting new location of initialisesettings to lsearch-global-2.1.conf so the incremental updater works again [production]
17:20 <Fred> srv224 is now an image_scaler. Adjusted on lvs3, ganglia and dsh's node_list. [production]
17:14 <Fred> db20 back online [production]
16:50 <Fred> rebooting db20 as it is in a "state" [production]
16:45 <brion> looks like we've lost internal /home NFS, which makes some of our internal services very unhappy. investigating... [production]
16:43 <brion> ganglia out. [production]
13:44 <apergos1> doing next round of removals on ms1 (/export/upload/wikipedia/en/thumb/2) to keep ahead of the game [production]
04:15 <apergos> starting removal of /export/upload/wikipedia/en/thumb/1 on ms1 (moved away and symlink to ms4 done already) for more space [production]
03:54 <brion> synchronized wmf-deployment/wmf-config/CommonSettings.php 'Disabling sitenotice from maintenance' [production]
03:29 <brion> reenabling uploads & image deletion/undeletion [production]
03:29 <brion> synchronized wmf-deployment/wmf-config/CommonSettings.php [production]
03:28 <brion> remounting ms1 on apaches [production]
00:49 <atglenn> only about 1gb gain on each so doing all of /export/upload/wikipedia/en/thumb/0 [production]