tlug: Re: root mirror (was: LILO Vs. 1024??)

[Rex Walters <rex@example.com>]

On Wed, Oct 07, 1998 at 12:39:34PM +0900, Stephen J. Turnbull wrote:

> Paranoid structure:
> 
> /dev/?da1	 50 MB		 /
> /dev/?da2	 128 MB		 <swap>
> /dev/?da3	 50 MB		 /.root_mirror
> 
> etc.  I've yet to have a LILO problem with this, I think that's
> because everything of interest to LILO always fits under the magic
> BIOS limit.  (But maybe somebody can tell me whether that's right or
> wrong; I could just be lucky.)

Right.

I also recommend this, but I caution against the word "mirror".  I
prefer the word "clone" or "backup".  That is I don't think you're
really referring to raid1 mirroring, are you?  I strongly recommend
doing periodic copies from /dev/?da1 to /dev/?da3 (via a backup program,
dd, or whatever) rather than actually doing raid1 mirroring.  

Most of the problems I've seen in this area come from operator error:
deleting or mangling an important file in the / filesystem.  Often you
don't notice the problem until months later when you reboot the
system for whatever reason.  Having a copy of / around can really make
your day in this situation.  If you're actually doing raid1, though,
deleting/mangling a file will corrupt *both* partitions, leaving you
with an unbootable machine.

Out of curiosity, how are you populating the /.root_mirror filesystem?
My approach would probably be to unmount it, dd from ?da1 to ?da3 when
you "know" (hmmphf) nothing is being modified in /, then fsck ?da3 to be
sure and remount.

Ideally, I'd also prefer to keep /usr mounted read-only at all times and
keep a clone of it laying around as well.  Big disks are cheap these
days.

On the high-end fileservers my company sells, standard practice is to
periodically "clone" the entire "root disk" (a dedicated disk containing
*only* the OS and configuration info -- root, usr, and var, no user
data) to a separate "root clone" disk.  Since all drives are in hot-plug
cannisters, this made it really easy to physically exchange disks and
reboot from a known good image in the event of a problem.

This is particularly nice for OS upgrades.  Standard procedure for an
upgrade is to clone the root disk, run the update utility on the cd
(which uses chroot to ensure changes are *only* made to the root clone,
not the live root disk).  After all updates are applied, local
modifications/configuration is performed, etc., the system is shutdown,
root and root-prime are physically swapped, and the system is booted
with the new OS.  

The beauty is that backouts in the event of unforseen incompatibilities
are swift and GUARANTEED.  The original root disk was never touched, so
you KNOW you can get back to where you started from.  You just shutdown,
swap the disks back to the way they were, boot, and your back running
your original OS *EXACTLY* as you were before the upgrade.

Even when everything goes according to plan (usually the case ;-) it's
awfully nice to be able to mount the original root/usr/var partitions to
copy over some data you forgot about in the upgrade.

The only thing that's missing to be able to accomplish the same thing
with a Linux system is the ability to "clone" an ext2 filesystem
reliably.  On an Auspex, ax_clonefs will ensure that any modifications
that occur while the copy is running are handled correctly (there is
actually a brief lockout at the very end to ensure that all metadata is
synched on both sides correctly).  

You can use dd or whatever with an ext2 filesystem today, but if any
meta-data blocks are modified during the copy you will probably end up
with garbage.  Using a filesystem based utility like tar/cpio or cp
would help limit damage to individual files or directories rather than
wholesale filesystem corruption, but at the expense of longer copy
windows and other little gotchas ("holey files", unrestorable images,
etc.).

I'd REALLY like something like ax_clonefs for Linux.  

Regards,
-- 
Rex
---------------------------------------------------------------
Next Meeting: 10 October, 12:30 Tokyo Station Yaesu central gate
Featuring the IMASY Eng. Team on "IPv6 - The Next Generation IP"
Next Nomikai: 20 November, 19:30  Tengu TokyoEkiMae 03-3275-3691
---------------------------------------------------------------
Sponsor: PHT, makers of TurboLinux http://www.pht.co.jp