Re: [tlug] Apache: Logging: how-to pipe logs or log toanothermachine(wasDisk

[Tim Hurman <kano-tlug@example.com>]

On Fri, 6 Sep 2002, Godwin Stewart wrote:

> On Fri, 06 Sep 2002 16:03:38 +0900, "Jonathan Q" <jq@example.com>
> squashed this out of the keyboard:
>
> > >I currently get about 50GB of log files a day
> >
> > 50 *gig*?!  How many sites are you hosting?
>
> I'm glad I wasn't the only one to find that figure pretty large!
>
> 50 GB/day = roughly 600KB/second
>
> At an average of about 100 bytes per log line, that's 6000 hits/second.
>
> Tell me, what kind of computer can withstand that kind of battering so I can
> go and drool over it? :)
>

I dont gzip the log files as they go raw to disk. I then rotate the files
about twice a day (when they hit 2G) and bring them back over the private
network at night. There are seperate logging disks in a veriety of Sun
hardware E450/1405/V480 style machines. This is only one site:
http://www.bbc.co.uk/

The sister site (http://news.bbc.co.uk/) which I also get the logs for is of
a similar size, ie another 50G a day. It works out to be about this much
traffic:

http://support.bbc.co.uk/support/mrtg/internet/

Tim.

-- 
He who laughs last, laughs at 75 baud.
Tim Hurman - Email: kano at kano.org.uk - Phone: Yeah right.