Cloning Linux drives (notes for sio)

[siosios]

seeing as the drive in the server is showing signs of failure i am going to have to use this info to clone the drive. this is useful for more then me so im posting it here.


Last Modified: Aug/30/2001
Suggestion/Comments: [email protected]
Master Copy: http://www.rajeevnet.com/hacks_hints/os_clone/os_cloning.html

--------------------------------------------------------------------------------

Wonders of 'dd' and 'netcat' :: Cloning Operating Systems
Anytime we think about installing OS on more than one system 'cloning' comes to our attention. Because we are too lazy . Well that is one of the important characteristics of Systems Administrator so that he/she is forced to automate. In this document we will try to exploit the power of low level data transfer command popularly known as 'dd' and netcat. These programs are available for all major UNIX, Linux and Windows platforms. These commands are fairly popular among Forensics Analysis professionals.


Problem Description:

You got more than 1 machine with almost same hardware. i.e. same hard drive, SCSI card, graphics etc. You setup one single machine from top to bottom and now it is time to replicate OS setup on other machines. Commercial Software such as Ghost does a good job in cloning Windows based machines and now many of these software support Linux ext2 file system also. dd although very crude but gives you enough flexibility to manipulate cloning as you wish. We have demonstrated cloning of hard drives in machines loaded with Linux, Win2K, Solaris, HPUX machines using dd . This document is not a single click solution so you may have to juggle through here first. Once you get a hold of this process then it is very powerful to create your own disk cloning schemes save lots of time and hassle.

*more below*

[siosios]

Basic concept:
'dd' command can copy any data bit by bit from one location to another location. So a simple command
dd if=<src> of=<dst>
where, <src> and <dst> can be a file, file system partition or whole hard drive so anything which can be read/write in binary form, dd can handle it. dd however is not a network program. In order to support dd with networking feature we use another nice command 'netcat'. netcat can be used to connect any TCP/UDP servers and a very good tool for diagnostics also. A typical netcat can run both into client server mode. such as:
server% nc -l -p 30000 ==> (Listen for port 30000 on <server> )

client% nc <server> 30000 ==> (Connect to <server> at port 30000, ready to communicate)

This document will explain cloning under Linux, but concept is very similar for all other operating systems also for which 'dd' and 'netcat' binaries are available.


Operating System Cloning (Using STANDALONE machine):

Let us assume we have two drives (sda) and (sdb) attached to the system ( Example: Linux box, but can be any other OS). (sda) is drive with Master OS (let's call it Master OS drive) and (sdb) is a drive (slave drive) where we have to clone data from (sda).

IMPORTANT (IF you are cloning RAID/root devices): If slave drive has already RAID and/or root partition setup before. (especially / (root) partition), make sure you run fdisk (Use some 3rd Linux box if required) on slave hard drives , remove any partitions on slave hard drives and make it a plain new disk. If you don't do this and connect both drives with Master Linux box, it is very likely that Master linux box may come up with slave drive as (sda) i.e. primary device rather than expected (sdb) because how they appear in SCSI scan list. While following steps below you may destroy contents of actual Master Linux drive. Soyouhavebeenwarnedalready .

CAUTION (IF you are using SCSI drives) : In case slave drive is connected to external SCSI controllers and if external SCSI controllers appear first in SCSI scan list during boot and BIOS scan, then external drives will get first device name such as /dev/sda , /dev/sdb and your master Linux box may not boot from desired hard drive. In order to avoid this, DISABLE BIOS scan for external SCSI controller. This will cause not to scan SCSI drives connected with external SCSI controller, so Linux will get internal harddrive as /dev/sda and boot from desired disk. Drives on external controller will be available when Linux Kernel try to scan all SCSI bus while booting. For a typical Adaptec SCSI controller:
During Boot time, Go to SCSI controller BIOS screen (Ctrl-A)
Select Controller Configuration -> Configure/View Host Adapter Settings -> Advance Configuration options -> Host Adapter Bios = (Disabled: Not Scan) .
Save options and reboot box, During next boot, you will see SCSI BIOS Not Installed for this card.
DO NOT DISABLE BIOS SCAN FOR INTERNAL SCSI BUS. THIS IS NECESSARY TO FIND PRIMARY BOOT DISK.

Basically in any case IDE ot SCSI make sure you are booting from correct harddrive and that should appear as /dev/sda (Master drive) before you start any cloning process.

Let's assume we have to clone a harddrive (sda). Which has a partition table setup below. It has 1 NTFS partition loaded with WinNT/Win2K and rest Linux partition. (swap, Linux and Raid partition). Assuming second (slave) harddrive (sdb) is also attached to the same system.

Device Boot Start End Blocks Id System
/dev/sda1 1 9 72261 83 HPFS/NTFS
/dev/sda2 10 75 530145 82 Linux swap
/dev/sda3 76 467 3148740 fd Linux raid autodetect
/dev/sda4 468 2200 13920322+ 83 Linux

A simple way to clone this drive (/dev/sda) to another drive attached to this system (/dev/sdb) is to use dd command.

dd if=/dev/sda of=/dev/sdb

This command will copy each bit from sda (Master drive) to sdb (Slave drive) including MBR (Master Boot record). Thus after cloning new drive (sdb) is ready for deployment. This will also copy any information like File System IDs etc.

Since these days drive size is getting bigger and may run upto 100+ GB, this whole dd process may take long time and obviously there is no point in cloning Linux swap area or empty partitions which doesn't contain any useful data yet. Hence in this situation it is best to clone only relevant partitions. For this you need to partition second drive beforehand.

Note: Both drives are partitioned exactly same. If you have different brand harddrives, make sure each partition on second drive must be equal to or greater than first drive partitions. Also make sure File system ID should match for second drive also.


Device Boot Start End Blocks Id System
/dev/sdb1 1 9 72261 83 HPFS/NTFS
/dev/sdb2 10 75 530145 82 Linux swap
/dev/sdb3 76 467 3148740 fd Linux raid autodetect
/dev/sdb4 468 2200 13920322+ 83 Linux

Now cloning process partition by partition will look like:

First step is to copy MBR (Master Boot Record) to second drive. MBR is read right after BIOS in PCs bootstrap process. In case of Linux this will store LiLo ( Linux Loader) setup to find out Linux or Windows kernel. MBR is located with in first 446 bytes in harddrive (or partition) selected to store MBR during Linux install time or Windows install time. In our Linux installation we have selected MBR to be stored on first harddrive. Steps below will make second drive as a bootable drive.

Note: here we are using whole drive sda and sdb as input and output arguments of dd. (This process of making Solaris, HPUX drives bootable may be different but they allow you to setup boot record also just like PC's MBR)

dd if=/dev/sda of=/dev/sdb bs=446 count=1

There is no reason to clone swap partition. Swap is raw area.
Now you can clone other relevant partitions. Let's say /dev/sda3 will contain Linux OS and mounted as / (root) having ext2 or some other file system and /dev/sda4 is mounted as /home but doesn't contain any data. (May or may not contain any file system)

dd if=/dev/sda1 of=/dev/sdb1 ==> Clone NTFS partition
dd if=/dev/sda3 of=/dev/sdb3 ==> Clone RAID-1 partition having ext2 FS or some other.
So in this case we can save time just by cloning desired partitions.

*more below*

[siosios]

Operating System Cloning over network:

One major bottleneck in above process is we have to physically open boxes, connect harddrive to Master box and the run clone process. This is easier in case of desktops where you have a liberty to connect external drives (IDE, SCSI bus). But Laptop can hardly house one IDE drive in general and there are no easy way to open and connect second drive for cloning. Thus above process will be highly useful if cloning process can be used over network. There are several possible combination presented here. Idea here is we have Master Linux box up and running over network and we boot slave box having harddrive which is to be cloned but we use some alternate media such as boot CD and boot slave linux using root file system on CD itself *NOT* on harddrive so that we are free to write on slave hardrive.
Master Box-----------network-----------Slave box
[OS(Linux) up and running] [ Booting *NOT * using slave drive]
192.168.0.1 192.168.0.254

One of Following 3 methods can be used to boot slave box using alternative media.

Method [1] Making your own root filesystem on ext2 CDROM. (Not Scalable )

One can make a small Linux distribution (less than 650MB) which can fit into CDROM. Burn this CDROM with ext2 filesystem (not ISO9660) and then use Linux boot floppy to boot from and use CDROM ext2 file system as / (root) file system (read only) (instead of root file system on Harddrive). This process although is doable but has issues like you need to have all possible drivers for network, SCSI etc. Making your custom ext2 read only file system on CD and booting from it would be quite a trial and error issue. If you are interested in making such Cds or bootable CDs see reference section for links. I once did that to clone HP Omnibook 6000 laptops loaded with Linux+Win2K OS together and it worked pretty okay but this is not a scalable solution though.



Method [2] Using popular Linux distribution and floppy combination.

On a similar line Linux distribution such as RedHat/SuSe boot CDROM at OS install time will allow you to boot into some kind of rescue system. In case of RedHat boot from RedHat OS CD and at initial OS install prompt type 'linux rescue ' at the boot time and this will let you use CDROM as root file system and provide you a shell prompt. Linux distribution uses this facility to repair problematic Linux install but we will use this for getting just shell prompt. Great thing about this is most Linux distribution comes up with lots of popular SCSI, network drivers so you don't have to worry about cooking your custom bootable CD.

Many common utilities including 'dd' command usually available in rescue mode. However you need netcat (static binary not dynamically linked) command. You can download netcat distribution and recompile it as a static binary (use -static flag). When I compiled it is small enough to fit into one floppy. So you can copy this into floppy. (I formatted floppy in ext2 format and then mounted in Linux system, copied netcat binary there.)

mkfs /dev/fd0
mount /dev/fd0 /mnt/floppy
cp nc /mnt/floppy
umount /mnt/floppy

So with 'linux rescue' mode and netcat binary on floppy you can use dd and netcat to clone your system over network. As we will see below.



Method [3] Modifying popular Linux distribution CDs and recreating your personal bootable ISO image:

If for some reason netcat won't fit in 1 floppy or you need more utility/binaries. Then you can change Linux distribution (SuSe/RedHat CD). This is a little hack but works.

NOTE: ISO images are read-only file systems. Even if you have an iso image (Say by using dd command )
dd if=/dev/cdrom of=redhat-boot-cd.iso
and if you try to mount this iso file using loopback device with option read/write (-o rw) (you need to have loopback device support (CONFIG_BLK_DEV_LOOP=y) compiled in kernel to do that)
mount -o loop -o rw ./redhat-boot-cd.iso /mnt/cdrom
This won't allow you to write/modify ISO filesystem.

I haven't found any good solutions to edit iso image directly , One such tool is winISO (http://www.winiso.com ) this is a shareware package so you have to pay for it. But you can use this to add more files in your ISO image and burn new image back to new CD. If you know any better solution let me know also

Following steps are useful for adding additional files in RedHat bootable ISO image and burning a new CDs with additional files as of your choice.

First mount RedHat CD (say RedHat 7.1). mount /dev/cdrom /mnt/cdrom
Create a directory where you store stuff what goes on new CD. mkdir /home/cdburn
cd /mnt/cdrom
(tar cbf 20 - *) | (cd /home/cdburn | tar xvbf 20 -)
This tar command will copy whole CD (~650MB to your Harddrive). In most cases you do not need all CD. Important stuff you need from directories, dosutils/, images/, RedHat/base. But if you do not want to play much simply copy the whole CD as above tar command and then you can delete RedHat/RPMS directory. These are simply RPM packages and since our intention is not to install OS from CD so we don't need that.
Create subdirectory directory mkdir /home/cdburn/mystuff/ and add all your stuff such as static version of netcat binary etc. there.
Now delete all TRANSLATION TABLES (TRANS.TBLS) files, otherwise mkisofs command will complain. find ./ -name "TRANS.TBL" -exec /bin/rm {} \;
Make bootable iso image out of above distribution (/home/cdburn). Use mkisofs (part of mkisofs-1.9-6 package). The command below will create a bootable ISO image using initial boot image specified by -b, the -c option is for the boot catalog file. The -r option will make appropriate file ownership and modes. This iso image redhat-bootcd.iso is very similar to what is provided by RedHat except it will have our stuff also and we may have deleted any unnecessary contents such as RedHat/RPMS directory.
mkisofs -r -b images/boot.img -c boot.catalog -o /tmp/redhat-bootcd.iso ./
Finally burn this iso image redhat-bootcd.iso using your cd-burner.
Now the Real drill:
Whatever method you choose to boot slave machine ( RedHat bootable CD + floopy or custom bootable RedHat CD), ultimate aim is to obtain shell, dd and netcat binary after 'linux rescue'. After you get shell you can access files stored on boot CD by changing directory to /mnt/sources/mystuff .
Hopefully your ethernet card has been detected by now. (as most Linux distributions allow OS install over network) if not then you have to load drivers for your ethernet card. Linux distribution documentation usually tells that how and sometimes they provide extra drivers floppy. In case of RedHat these floppy images are generally stored under directory images/ and you can copy these images to your floppy using commands like
dd if=<floppy> of=/dev/fd0

*more below*

[siosios]

On Slave machine:
Run netcat command first on slave linux box (that to be cloned and booted using Linux boot CDROM as 'linux rescue' (See also Shell script case [1] in automation section below). Once ethernet card has been detected. (Use ifconfig -a command to check) assign IP address to this interface now on slave machine. Define loopback interface also. (You may choose different IP address for eth0). Also you may need to define /etc/hosts file before you can assign IP address. Use following commands to create your new /etc/hosts. (These are actually created in ram file system RAMFS).

rm /etc/hosts
echo "127.0.0.1 localhost" > /etc/hosts
echo "192.168.0.254 fakehost" >> /etc/hosts

ifconfig lo 127.0.0.1 up
ipconfig eth0 192.168.0.254 up

Assuming Master Linux box (from where you want to clone) is up and running with IP 192.168.0.1.

slave% nc -l -p 9000 | dd of=/dev/sda (Replace /dev/sda with actual drive on your slave machine)

This will listen at port 9000 and whatever it gets at port 9000 will hand over to dd command and dd will simply write that to on slave harddrive (sda) bit by bit. Here I am assuming dd and netcat (nc) are available either through floppy (/mnt/floppy/nc or through /mnt/sources/mystuff/nc). In case of floppy you need to mount floppy first using command:
mount /dev/fd0 /mnt/floppy


On Master machine:
Now Login on master linux box and run following command. (It is advisable that Master Linux box should be in calm state , i.e no major jobs running on the machine). This command below will read master disk bit by bit and throw this bit stream to netcat command which is connected to netcat command at port 9000 on <slave> box.

master% dd if=/dev/sda | nc 192.168.0.254 9000

That's it. You may have to wait for long time depending upon network speed and size of your harddrive. Typically 36GB drive may take 50 minutes over 100Mbps link. Again rather than cloning complete drive we can clone only relevant partitions and MBR only. That will make cloning much faster like we saw in above section.



*more below*

[siosios]

Automating process and Case studies:
One of the primary reason for using dd and netcat way of cloning OS instead of using commercial software such as Ghost is we have a liberty to automate process as we like. Following scripts may help in automating cloning process.
Case [1]: Script for Slave machine (netcat and dd cloning) on the fly.

Make sure you have netcat command available either /mnt/floppy or /mnt/sources/mystuff area.
=================================================
cloneme.sh :: Shell script for slave machine.
=================================================
#!/bin/sh
############### Edit variables below ######################
FLOPPY_PATH=/mnt/floppy
MYSTUFF_PATH=/mnt/sources/mystuff

# Uncomment only One of the options below.
#### OPTION ==> 1 if using floppy ################
#NC=$FLOPPY_PATH/nc
#### OPTION ==> 2 if using mystuff/ on CD #########
NC=$MYSTUFF_PATH/nc

LPORT=9000
DEST=/dev/sda
SRC=$DEST
############# No need to edit after this in general ###########

if [ $# -eq 1 ]
then
IPADDR=$1
echo "################################################# ##############"
echo " If there are no errors here. You need to run following"
echo " command on Master Box."
echo ""
echo "dd if=$SRC | nc $IPADDR $LPORT"
echo "################################################# ##############"

echo ""
echo "##>> Preparing /etc/hosts ##"
rm /etc/hosts
echo "127.0.0.1 localhost" > /etc/hosts
echo "$IPADDR fakehost" >> /etc/hosts

echo "#================================================ ===================="
echo "NOTE:: If you need to create routes"
echo " #route add -net <DEST_NET> netmask 255.255.255.0 gw $IPADDR metric 0"
echo "#================================================ ===================="

echo "##>> Preparing interfaces lo and eth0 ##"
ifconfig lo 127.0.0.1 up
ifconfig eth0 $IPADDR up

echo ""
echo ">>> Now start listening(at $LPORT) for traffic from Master "
echo "$NC -l -p $LPORT | dd of=$DEST"
$NC -l -p $LPORT | dd of=$DEST

echo ""
echo "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%"
echo " Cloning Process completed..... Reboot Now"
echo "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%"

else
echo "Usage:: cloneme <IP_ADDR_OF_THIS_MACHINE>"
fi




Case [2] Saving Disk Images (Export Image for later use):

Although you can clone running machine over the network anytime. But it is sometimes desirable to store base installation as a reference image and you may want to clone from this pristine image later. With the help of dd you can image disks also. But let's discuss some issues first.
Most 32 bit operating system (Linux for IA32 , Windows etc.) will have physical limitation on max file size. In general practical limit is 2GB as a max. file size. 64 bit OS (Solaris8, HPUX 11.0, Linux for IA64, etc.) will not have this limitation. So if you use dd to copy harddrive image you can maximum image 2GB harddrive. That is pretty useless these days. Fortunately dd can image in chunks and you can specify start and end blocks, skip blocks etc. So idea here is to image your big harddrives in chunks of approx. 2GB files over network. Although I noticed RedHat 7.1 with Linux 2.4.x kernels will allow fie size even bigger than 4GB on ext2 FS.
Also if you want to store images in compressed format (to save space) it is desirable to have each image file size not too large.
Following perl script (export-image.pl) can be used to image local Linux harddrive /dev/hda to remote machine over NFS using dd. If you are not running NFS you can implement same thing using dd and netcat. For now that would be a manual process. If somebody knows a better way to run netcat and transfer multiple files automatically between two machines please let me know and I will cook up some automation script here.

This perl script is actually use dd command something as described below. This is imaging your big harddrive into chunks of 1950 MB files named (1, 2, 3, 4, .....) over NFS to remote machine.
($NFS is NFS destination directory on another server having plenty of space)

For 1st Image:
dd if=/dev/hda of=$NFS/1 bs=1024k count=1950 skip=0
For 2nd image: (Skipping the part of harddrive used for 1st image.)
dd if=/dev/hda of=$NFS/2 bs=1024k count=1950 skip=1950
For 3rd image: (Skipping the part of Harddrive used for 1st+2nd image)
dd if=/dev/hda of=$NFS/3 bs=1024k count=1950 skip=3900
and so on.


*more below*

[siosios]

In case you want to use netcat you can simply pipe above dd commands manually to netcat and listen using netcat and dd on remote machine, just like we used netcat and dd to clone hardisks above. For example imaging harddrive on machineA and saving image on machineB.

For 1st image:
machineB% nc -l -p 9000 | dd of=1
machineA(master)% dd if=/dev/hda bs=1024k count=1950 skip=0 | nc machineB 9000
For 2nd image:
machineB% nc -l -p 9001 | dd of=2
machineA(master)% dd if=/dev/hda bs=1024k count=1950 skip=1950 | nc machineB 9001
For 3rdimage:
machineB% nc -l -p 9002 | dd of=2
machineA(master)% dd if=/dev/hda bs=1024k count=1950 skip=3900 | nc machineB 9002
and so on.

Once you have images (1, 2, 3, 4 ....) stored on network then you can boot your slave Linux box using bootable CD and pull these images to slave box as described in case [3].

================================================== ======
export-image.pl :: Perl script to image big harddrive using dd and NFS.
================================================== ======
#!/usr/bin/perl
################################################## ###
#This script will run dd command (in serial) and dump
#1950 blocks (1.9GB) file for each.
#Run script as perl export-image.pl
################################################## ###

################ Edit variables below #########################
#device is raw device name for harddrive to be cloned (imaged).
$device="/dev/hda";
#mount NFS file system with large space available which can hold images.
$nfs_path="/nfs/remote/home/tmp";
#Image name (read from user) (Make sure you have $nfs_path/$image directory)
#on remote machine.
$image="ob6000";
################################################## ##########
$dd="/bin/dd";
#For compressing image
$bzip2="/usr/bin/bzip2";
$suffix=".bz2";
############## No need to edit after this #########################

$bs="1024k";
$block_count=1950;
$image_dir="$nfs_path/$image";
$compress=$bzip2;

$proceed=0;

if(!(-d $image_dir) )
{ die "\nOops!! Image Directory $image_dir must exist with chmod 777 permission\n"; }

system("clear");
print <<MSG1;
################################################## #########
NOTE:: COMPRESSION TAKE TOO MUCH TIME(Many HOURS) OVER NFS.
So better compress manually latter on server itself.
################################################## #########
\n\n Do you want to compress images using $compress [y/n] (Default n) = \t
MSG1

$compress_flag=<STDIN>;
if(($compress_flag eq "y") or ($compress_flag eq "Y"))
{ $compress_flag=1; }
else
{ $compress_flag=0; }

print "\n\n";
print "************************************************* **\n";
print " Local Device = $device [SOURCE] \n";
print " Image Dir = $image_dir [TARGET] \n";
print "************************************************* **\n\n\n";
print "Dude! I hope you understand what are you doing by pressing [y/Y] here \n";
print " Press [y/Y] if you want to continue .. ";
$con=<STDIN>; chomp($con);


if(($con eq "y") or ($con eq "Y"))
{
$i=0;
$image_size=1; #Some fake value greater than zero.


print "\n\nDisk Imaging starts...\n";
system("date");
while($image_size > 0)
{
$image_name="$image_dir/$i";
print "##############################################\n" ;
print "Creating Image $image_name\n";
print "##############################################\n" ;
$skip=$i*$block_count;
print "$dd if=$device of=$image_name bs=$bs count=$block_count skip=$skip \n";
system("$dd if=$device of=$image_name bs=$bs count=$block_count skip=$skip");
if($compress_flag)
{
print "Compressing Image: $bzip2 $image_name => $image_name$suffix\n";
system("$bzip2 $image_name");
$image_name .= "$suffix";
}
++$i;
$image_size=(stat($image_name))[7];
system("date");
}
}
else
{
print "Bye Bye ...\n";
}




*more below*