401-450 of 1529 results (5ms)
2009-02-23 §
03:42 <Tim> rsync done, uploads re-enabled, b/c symlinks set up [production]
03:37 <Tim> doing rsync [production]
03:31 <Tim> temporarily disabled file uploads on all private wikis, for migration to ms1 [production]
02:50 <Tim> same for commons ForeignDBViaLBRepo directory, ScanSet directory, CentralNotice directory, [production]
02:44 <Tim> fixed CommonSettings.php location of deleted images, upload3 -> upload5, appears to have been moved already [production]
2009-02-21 §
19:49 <mark> Installed gmond on eiximenis [production]
19:02 <domas> db26 lacks 8g of ram :) [production]
19:00 <mark> Restarted stuck apache on srv217 [production]
17:26 <mark> Started apache on srv218-221 [production]
17:24 <mark> Restarted stuck apache on srv217 [production]
17:07 <mark> Squid/kernel upgrade complete [production]
16:46 <mark> Increased max-connections per upload squid to ms1 to 100 [production]
15:58 <mark> Running automated upgrade/reboot of squid and kernel on sq43-47 [production]
15:58 <mark> Upgraded squid and kernel on sq41-42, sq48-50, and rebooted [production]
15:44 <mark> Upgraded squid and kernel on sq36-40, and rebooted [production]
12:55 <river> fixed reverse dns entries for ms3/ms4, which had got swapped somehow [production]
11:55 <Tim> re-enabled ExtensionDistributor [production]
11:16 <Tim> removed syslog.0 and messages.0 on srv170 and srv176, they had critical disk free on / [production]
03:25 <Tim> started apache on the image scaling servers [production]
02:51 <brion> ran sync-common on srv199 while i'm at it [production]
02:48 <brion> zeroing out stupid giant syslog files on srv199 [production]
02:46 <brion> srv199 is out of disk space [production]
02:46 <brion> copying hacked-up copies of InitialiseSettings/CommonSettings back to /home so the changes aren't lost this time [production]
02:23 <mark> db20 back up, for reals [production]
02:19 <mark> Rebooting db20 with upgraded RAID controller firmware [production]
02:13 <domas> flashing BIOS helped [production]
02:13 <mark> db20 up! [production]
02:04 <brion> services on bart (secure, planet) are temporarily offline while server is poked at [production]
01:50 <brion> seeing pages, yay [production]
01:49 <brion> running apache2ctl start or apachectl start for various apaches [production]
01:47 <domas> I FOUND HOW TO REVIVE APACHES [production]
01:46 <brion> think i killed em, now trying to restart apache procs [production]
01:43 <brion> poking to see if we can restart apaches... [production]
01:42 <brion> syncing fixed InitialiseSettings/COmmonSettings to apaches [production]
01:14 <brion> and flyingparchment [production]
01:14 <brion> domas and mark are attempting to restart the NFS server, but aren't mentioning any details in the public channel or log [production]
00:52 <domas> http://p.defau.lt/?_M1iGbA0PCz2OOt2_KKPug [production]
00:52 <mark> db20 in trouble [production]
00:39 <mark> @brion you don't need to wake up [production]
00:36 <domas> disabled 2006 fundraising cronjob on amane :-) [production]
2009-02-20 §
23:31 <Rob> upgraded squid and kernel on sq34-sq36 [production]
23:12 <Rob> upgraded kernel and squid on sq31-sq33, redeployed and online [production]
23:08 <brion> updating CentralNotice for improved test script (plus i8n update) [production]
22:54 <Rob> upgraded kernel + squid on sq28-sq30 [production]
22:29 <Rob> completed upgrades to sq25-sq27 [production]
22:12 <Rob> upgrading kernel and squid versions on sq25-sq27 (if i crash the site, i apologize in advance) [production]
22:08 <Rob> upgraded kernel and squid on sq24 [production]
21:59 <river> added current patches to ms4, set zil_disable=1 and rebooted [production]
21:30 <brion> srv31 seems to be down, so no dump activity [production]
21:08 <brion> scapping to update FlaggedRevs to r47588 (fixing fatal err) [production]