701-750 of 2598 results (6ms)
2009-04-24 §
17:38 <Rob> srv57 online [production]
17:26 <Rob> reinstalling srv58 [production]
17:16 <mark> Set up switchport for srv57 on asw-c4-pmtpa [production]
17:10 <Rob> reinstalling srv57 [production]
17:08 <Rob> srv75 back online [production]
17:02 <Rob> srv74 back online [production]
16:47 <Rob> srv73 back online as apache [production]
16:45 <robh> synchronized php-1.5/mc-pmtpa.php 'removed srv75' [production]
16:43 <Rob> srv71, srv72 back online as apache [production]
16:43 <Rob> taking down srv75 for reinstall [production]
16:37 <fvassard> synchronized php-1.5/mc-pmtpa.php 'swapping out srv72 for srv100 and srv73 for srv101 while srv[72,73] are being ubuntified' [production]
16:26 <Rob> srv72, srv73, and srv74 down for reinstallation [production]
16:23 <root> synchronized php-1.5/mc-pmtpa.php 'swapping out srv71 for srv70 and srv74 for srv92 while srv[71,74] are being ubuntified' [production]
16:05 <Rob> srv34 back online reinstalled as ubuntu [production]
16:04 <Rob> reinstalling srv71 [production]
16:04 <Fred> restarted apache on srv99 [production]
15:21 <Rob> srv34 coming down for reinstall [production]
15:13 <Rob> amane reinstalled for tomasz [production]
14:59 <Rob> amane reinstall started [production]
14:36 <rainman-sr> search9,10 also up; everything should be normal again [production]
14:33 <Rob> amane shutting down for rain controller work [production]
14:27 <rainman-sr> search5-8 back in search pool [production]
14:16 <Rob> shutting down search9 & search10 for memory upgrade [production]
14:15 <Rob> search7 & search8 memory upgraded, systems rebooted [production]
14:07 <Rob> search5 and search6 back online. [production]
14:05 <Rob> memory upgrade complete on search5 & search6, rebooted. [production]
14:02 <rainman-sr> done with initial index warmup on search3,4, back in rotation [production]
13:59 <Rob> search5, search6 shutdown for memory upgrade [production]
13:58 <Rob> search4 memory upgraded and system back online [production]
13:55 <Rob> search3 ram upgraded and system is back online [production]
13:50 <Rob> search3 upgraded, rebooting. [production]
13:44 <Rob> shutdown search3 & search4 for memory upgrades [production]
07:18 <tstarling> synchronized php-1.5/db.php [production]
03:35 <andrew> synchronized php-1.5/InitialiseSettings.php [production]
03:35 <Andrew> Deployed AbuseFilter to fiwiki [production]
02:51 <tstarling> synchronized php-1.5/mc-pmtpa.php [production]
02:46 <Tim> srv127 has corrupted root partition, needs reinstall or repair. Shut down with echo o > /proc/sysrq-trigger. [production]
02:36 <tstarling> synchronized php-1.5/mc-pmtpa.php [production]
02:31 <Tim> killed srv124 with /proc/sysrq-trigger. Was very slow on ssh and was giving odd 403 errors via HTTP. [production]
02:21 <tstarling> synchronized php-1.5/README [production]
02:12 <andrew> synchronized php-1.5/CommonSettings.php [production]
02:10 <Andrew> srv127: rsync: mkstemp "/apache/common/php-1.5/.CommonSettings.php.TRNqkG" failed: Read-only file system (30) [production]
01:15 <tstarling> synchronized php-1.5/db.php [production]
01:14 <Tim> depooled db3 so that it can finish doing the querycache update without making lots of people wait for a MASTER_POS_WAIT [production]
01:03 <tstarling> synchronized php-1.5/InitialiseSettings.php [production]
01:03 <Tim> blacklisted Wantedtemplates on enwiki, has been running for more than a day. [production]
00:54 <Tim> restarting trackBlobs.php on hume for afwiki and enwiki [production]
2009-04-23 §
19:05 <brion> donate.wikipedia.org redirect borked, going to civicrm instead of public donation pages. server config needs updating [production]
16:54 <brion> db3 was lagging a bit; 403s a few minutes ago. catching up nicely now [production]
14:46 <robh> synchronized php-1.5/InitialiseSettings.php 'Added namespaces to huwikisource per bug 18557' [production]