Wednesday, October 30, 2019

Cloning a RHEL/CentOS 7.x onto a bootable second disk

This is a little one of those things where you just have to do it yourself.

Basically the requirement would be to have two disks on your system. I am testing this on an Oracle VM Virtualbox environment, with a CentOS 7.7 (1908) as a VM guest.

The rest of the VM guest setup is pretty straightforward nothing out of the ordinary, with two disks of the same size (or virtual disks) attached. This requires further testing done on a bare metal x86 server, which I would hope to be able to do soon with EFI.

The first disk is /dev/sda, and the second disk is /dev/sdb.

The second disk is basically empty. And obviously the first one isn't. And this is how the partition table looks like:

# parted /dev/sda p
Model: ATA VBOX HARDDISK (scsi)
Disk /dev/sda: 42.9GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  1075MB  1074MB  primary  xfs          boot
 2      1075MB  34.4GB  33.3GB  primary               lvm

Partition 1 (or /dev/sda1) is a native partition, where as partition 2 (/dev/sda2) is under LVM. As you can see here:

# pvs
  PV         VG      Fmt  Attr PSize   PFree
  /dev/sda2  centos  lvm2 a--  <31 .01g="" 8.00m="" p="">
# vgs
  VG      #PV #LV #SN Attr   VSize   VFree
  centos    1   8   0 wz--n- <31 .01g="" 8.00m="" p="">
# lvs
  LV            VG     Attr       LSize Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  home          centos -wi-ao---- 5.00g
  root          centos -wi-ao---- 5.00g
  swap          centos -wi-ao---- 2.00g
  tmp           centos -wi-ao---- 3.00g
  var           centos -wi-ao---- 5.00g
  var_log       centos -wi-ao---- 5.00g
  var_log_audit centos -wi-ao---- 3.00g
  var_tmp       centos -wi-ao---- 3.00g


The requirement is to basically to clone the contents of first disk (/dev/sda) to the second disk (/dev/sdb) which should include partition table, MBR or gpt, regardless.

And all filesystems are xfs, which is the default filesystem for all RHEL or CentOS 7.x install.

The steps

The fastest way to achieve this is to use the good ol' dd. I am lazy so, here's the command:

# dd if=/dev/sda of=/dev/sdb

You can add bs=512, or bs=1024 to make the dd cloning go faster (This is another topic altogether)

After cloning is complete, use parted to see if the exact same partition table is on the second disk (/dev/sdb). By right, you should be seeing:

# parted /dev/sdb p
Model: ATA VBOX HARDDISK (scsi)
Disk /dev/sdb: 42.9GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  1075MB  1074MB  primary  xfs          boot
 2      1075MB  34.4GB  33.3GB  primary               lvm

First things, first, test to see if /boot or /dev/sda1 is mountable. Before you do this you'll need to use xfs_repair (to zero the logs) in order to make it mountable:

# xfs_repair -L /dev/sda1

After that, the xfs file system on /dev/sda1 uuid needs to be regenerated because of the clone, where all uuid for the filesystems, PV and VG will be the same (duplicated) and only one can be activated at one time.

To do this:

# xfs_admin -U generate /dev/sda1

Once this is done, perform a test mount:

# mkdir -p /a/boot
# mount  /dev/sda1 /a/boot

Examine it's contents, by running :

# ls -l /a/boot
total 124568
-rw-r--r--. 1 root root   152980 Sep 30 22:23 config-3.10.0-1062.1.2.el7.x86_64
-rw-r--r--. 1 root root   151923 Jun 19 00:39 config-3.10.0-957.21.3.el7.x86_64
drwxr-xr-x. 3 root root       17 Jun 27 23:18 efi
drwxr-xr-x. 2 root root       27 Jun 27 23:19 grub
drwx------. 5 root root      132 Oct 29 23:56 grub2
-rw-------. 1 root root 57160365 Jun 27 23:24 initramfs-0-rescue-7ff3c24ba1924fe1af8443c0ea440407.img
-rw-------. 1 root root 21143881 Oct  3 18:41 initramfs-3.10.0-1062.1.2.el7.x86_64.img
-rw-------. 1 root root 21125579 Oct  3 18:42 initramfs-3.10.0-957.21.3.el7.x86_64.img
-rw-r--r--. 1 root root   318717 Sep 30 22:23 symvers-3.10.0-1062.1.2.el7.x86_64.gz
-rw-r--r--. 1 root root   314128 Jun 19 00:39 symvers-3.10.0-957.21.3.el7.x86_64.gz
-rw-------. 1 root root  3595191 Sep 30 22:23 System.map-3.10.0-1062.1.2.el7.x86_64
-rw-------. 1 root root  3545891 Jun 19 00:39 System.map-3.10.0-957.21.3.el7.x86_64
-rwxr-xr-x. 1 root root  6639904 Jun 27 23:24 vmlinuz-0-rescue-7ff3c24ba1924fe1af8443c0ea440407
-rwxr-xr-x. 1 root root  6734016 Sep 30 22:23 vmlinuz-3.10.0-1062.1.2.el7.x86_64
-rwxr-xr-x. 1 root root  6643904 Jun 19 00:39 vmlinuz-3.10.0-957.21.3.el7.x86_64

Unmount /a/boot.

# umount /a/boot

Perform the exact same thing with xfs_repair and xfs_admin on the LVs sitting on PV /dev/sdb2, which was cloned from /dev/sda2.

But before you can do that, you need to perform a VG import, but in this case, you will not be able to do so, because the uuid on the VG would have been duplicated (because of the dd clone).

Instead of using vgimport, use vgimportclone instead:

# vgimportclone --import -n centos1 /dev/sdb2

Where -n centos1 specifies the name of the new VG, because the original VG on /dev/sda2 (the source of the clone) is centos. /dev/sdb2 is it's PV.

Once you've done the above, you should be able to see both PVs and both VGs:

# pvs
  PV         VG      Fmt  Attr PSize   PFree
  /dev/sda2  centos  lvm2 a--  <31 .01g="" 8.00m="" p="">  /dev/sdb2  centos1 lvm2 a--  <31 .01g="" 8.00m="" p="">
# vgs
  VG      #PV #LV #SN Attr   VSize   VFree
  centos    1   8   0 wz--n- <31 .01g="" 8.00m="" p="">  centos1   1   8   0 wz--n- <31 .01g="" 8.00m="" p="">
Activate the VG centos1 if you are unable to access the LVs under it.

# vgchange -ay centos1
  8 logical volume(s) in volume group "centos1" now active


Once activated, you should be able to see all the LVs on both centos and centos1 VGs.

# lvs
  LV            VG      Attr       LSize Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  home          centos  -wi-ao---- 5.00g
  root          centos  -wi-ao---- 5.00g
  swap          centos  -wi-ao---- 2.00g
  tmp           centos  -wi-ao---- 3.00g
  var           centos  -wi-ao---- 5.00g
  var_log       centos  -wi-ao---- 5.00g
  var_log_audit centos  -wi-ao---- 3.00g
  var_tmp       centos  -wi-ao---- 3.00g
  home          centos1 -wi-a----- 5.00g
  root          centos1 -wi-a----- 5.00g
  swap          centos1 -wi-a----- 2.00g
  tmp           centos1 -wi-a----- 3.00g
  var           centos1 -wi-a----- 5.00g
  var_log       centos1 -wi-a----- 5.00g
  var_log_audit centos1 -wi-a----- 3.00g
  var_tmp       centos1 -wi-a----- 3.00g

The next thing for us to do is to ensure the LVs under the VG centos1 is mountable:

# for lvname in home root tmp var var_log var_log_audit var_tmp; do
> xfs_repair -L /dev/mapper/centos1-${lvname}
> xfs_admin -U generate /dev/mapper/centos1-${lvname}
> done

Once this is done, you should be able to mount these LVs. Remember the /a directory we created earlier. Mount all the relevant filesystems onto /a

# mount /dev/mapper/centos1-root /a
# mount /dev/sdb1 /a/boot
# mount /dev/mapper/centos1-home /a/home
# mount /dev/mapper/centos1-tmp /a/tmp
# mount /dev/mapper/centos1-var /a/var
# mount /dev/mapper/centos1-var_tmp /a/var/tmp
# mount /dev/mapper/centos1-var_log /a/var/log
# mount /dev/mapper/centos1-var_log_audit /a/var/log/audit
# mount -t proc none /a/proc
# mount -o bind /sys /a/sys
# mount -o bind /dev /a/dev

Chroot the /a by running

# chroot /a

Next find out what is the existing uuid for /dev/sda1 and /dev/sdb1.

# blkid | grep sd[ab]1
/dev/sda1: UUID="0f7eaa06-5707-45cc-99db-f3e507c44bcf" TYPE="xfs"
/dev/sdb1: UUID="9b6d885a-ea6d-4c28-997c-fb6f04dcec5e" TYPE="xfs"

Edit the contents of /a/etc/fstab, replace the UUID="<16 digit="" hexadecimal="">" from /dev/sda1 to /dev/sdb1 from the output above.

Change this:
UUID=0f7eaa06-5707-45cc-99db-f3e507c44bcf /boot                   xfs     defaults        0 0

To this:
UUID=9b6d885a-ea6d-4c28-997c-fb6f04dcec5e /boot                   xfs     defaults        0 0

Edit the contents of /a/boot/grub2/grub.cfg and locate the line containing:

### BEGIN /etc/grub.d/10_linux ###
menuentry 'CentOS Linux (3.10.0-1062.1.2.el7.x86_64) 7 (Core)' --class centos --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-3.10.0-1062.1.2.el7.x86_64-advanced-18586e25-cb00-4ef5-b913-e0613ede485a' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_msdos
        insmod xfs
        set root='hd0,msdos1'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 --hint='hd0,msdos1'  0f7eaa06-5707-45cc-99db-f3e507c44bcf
        else
          search --no-floppy --fs-uuid --set=root 0f7eaa06-5707-45cc-99db-f3e507c44bcf
        fi
        linux16 /vmlinuz-3.10.0-1062.1.2.el7.x86_64 root=/dev/mapper/centos1-root ro crashkernel=auto rd.lvm.lv=centos1/root rd.lvm.lv=centos1/swap rhgb quiet
        initrd16 /initramfs-3.10.0-1062.1.2.el7.x86_64.img
}

Replace the uuid marked in yellow to 9b6d885a-ea6d-4c28-997c-fb6f04dcec5e

Amend the vg name from centos to centos1 in the file /a/etc/default/grub. Ensure these lines containing:

GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet"

is changed to :

GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=centos1/root rd.lvm.lv=centos/swap rhgb quiet"

Update the grub.cfg file under /a/etc/grub2/grub.cfg 
# grub2-mkconfig -o /boot/grub2/grub.cfg

Perform a touch relabel to relabel all selinux context.
# touch /.relable

Exit from the chroot /a environment:

# exit

And now you should be back to the environment on the first disk.

Remember the /a/boot/grub2/grub.cfg entry you modified under the chroot /a environment ?

Use the same entries and edit the first disk's /boot/grub2/grub.cfg

And add the following entry, this entry is similar to what we have on the second disk (which we chroot /a into earlier)

menuentry 'CentOS Linux (3.10.0-1062.1.2.el7.x86_64) 7 (Core) on Disk 2' --class centos --cla                                                                           ss gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-3.10.0-1062                                                                           .1.2.el7.x86_64-advanced-18586e25-cb00-4ef5-b913-e0613ede485a' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_msdos
        insmod xfs
        set root='hd1,msdos1'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos                                                                           1 --hint-baremetal=ahci0,msdos1 --hint='hd0,msdos1'  9b6d885a-ea6d-4c28-997c-fb6f04dcec5e
        else
          search --no-floppy --fs-uuid --set=root 9b6d885a-ea6d-4c28-997c-fb6f04dcec5e
        fi
        linux16 /vmlinuz-3.10.0-1062.1.2.el7.x86_64 root=/dev/mapper/centos1-root ro crashker                                                                           nel=auto rd.lvm.lv=centos1/root rd.lvm.lv=centos1/swap rhgb quiet
        initrd16 /initramfs-3.10.0-1062.1.2.el7.x86_64.img
}

Save the file /boot/grub2/grub.cfg. And finally reboot to test.

Once reboot, your grub menu should look like this:


And all the above steps can be scripted. No problem.