Re: [tlug] Raid5 box & backup

[Christian Horn <chorn@example.com>]

On Mon, May 26, 2008 at 11:18:10AM +0900, Curt Sampson wrote:
> On 2008-05-26 03:58 +0200 (Mon), Christian Horn wrote:
> 
> > The journaling filesystem is another layer and stores everything in the
> > container-file while nfs/smb would work filewise.
> 
> You seem to consider "extra layers" to be a more than a very, very
> minimal potential source of file read/write errors, which is certainly
> not the case.

Ah, was just considering to implement those layers on the users computers
over the provides nfs/smb instead of just the two storage-boxen. 
Had the complexity of the setup on each operating system in mind, 
thats simpler using the 2 boxes.


> Consider that the journalling filesystem has much better recovery than
> NFS or SMB when a network connection is broken in the middle of a file
> update.
> 
> > Indeed, nice idea, that way rsync would finish faster. The "combine
> > blocks"-software had to be evailable on mac/win/linux in this case
> > thou....
> 
> No, you can build a dozen block devices on the server, concatenate them
> to one block device on the server, and export that one block device over
> the network.

Ok, had just the users computer in mind.


> > Also rsync could probably be reworked to handle big file in breaking 
> > them into pieces and checksuming/transferring if needed those.
> 
> Rsync alredy does exactly that; it's part of the whole point of rsync.
> However, as I pointed out earlier, rsync has to read all of every file
> with a changed timestamp, on both ends of the connection, with every
> sync. If that file is a hundred gigs in size, that means at a minimum
> 100 GB of reads and hashes at each end. With a modern disk, assuming the
> hash uses minimal CPU, that's still a good 30 minutes of clock time, not
> to mention trashing the I/O performance of the system for that time.

Right, thats checking modify-time of files vs. grokking their data to 
get the hash.


Christian