Two-Disk Layout for VMs

If you are an administrator and you manage multiple Debian VMs then you might have heard of this problem:

DISK CRITICAL - free space: / 2474 MB (10% inode=73%);

But what to do? Resizing the disk can be done while the VM is running, but usually the Linux kernel will prevent a reload of the partition table from which it booted from. The solution for this is to use a disk without a partition table, so the kernel cannot complain. This is difficult, if you deploy your machines using a preseed, because the debian autoinstaller does not allow a setup without a partition table.

Two-Disk Layout to the Rescue

If you set up your VMs with a two disk layout as follows:

  • SCSI:0:0 with 10MB size and a partition table
  • SCSI:0:1 with the real disk without a partition table

Then you can save the ordeal of booting every time you increase your disk size, as the kernel has no problems to rescan a device if there is no partition table on it! Of course, it's always a good idea to have lvm on the system disk.

A simple script that scans all disks for changes and modifies the found physical volumes for you:

#!/bin/sh
for scsi_device in /sys/class/scsi_device/*/device/rescan; do
    echo 1 > ${scsi_device}
done
for sdev in `dmesg | grep 'capacity change' | awk -F ] '{print $2}' | cut -f1 -d: | tr -d ' '` ; do
    pvdisplay "/dev/${sdev}" > /dev/null 2>&1
    if [ $? -eq 0 ];then
        pvresize "/dev/${sdev}"
    else
        echo "No physical volume /dev/${sdev} found!"
    fi
done

Of course, there are some improvements to the script, but it works quite nice in production. You can then go on and resize your logical volumes according to your needs.

Partman Hack

For this to work you need:

  • An OS that uses partman / preseed
  • A working PXE environment

One of the uglier things I have ever built is a hack that circumvents debians partman, so a normal preseed can work with the two disk layout. Let's say you have a preseed file that specifies the disk to be set up with lvm and the following partitions:

  • vg-root 2.5G
  • vg-var 1G
  • vg-var+log 1G
  • vg-tmp 1G
  • vg-swap 1G

Then you can use the following script, which I have used in production since debian wheezy. Remark: For debian bullseye and forward you will have to pull the fdisk udeb in a preseed or else this will fail!

#!/bin/sh
# this hack has to be in sync with the partman config in the preseed
sleep 5
# remove the partition table of /dev/sda, it should not exist on a clean install
dd if=/dev/zero of=/dev/sda bs=512 count=1
# delete swap
sleep 1
swapoff /dev/mapper/vg-swap_1
# remember to specify the correct volume group !
vgremove -ff vg
pvremove /dev/sdb1
pvremove /dev/sdb
sleep 1
dd if=/dev/zero of=/dev/sdb bs=512 count=1
sleep 1
# reread the partition table
blockdev --rereadpt /dev/sda
sleep 1
blockdev --rereadpt /dev/sdb
sleep 1
# only partition /dev/sda
echo -e "o\nn\np\n1\n\n\nw\n" | fdisk /dev/sda
sleep 1
pvcreate /dev/sdb
# force yes for the case that old vgs are found
# modify the sizes to your needs
echo -e "y\n" | vgcreate vg /dev/sdb
echo -e "y\n" | lvcreate -L 2.5G vg -n root
echo -e "y\n" | lvcreate -L 1G vg -n var
echo -e "y\n" | lvcreate -L 1G vg -n tmp
echo -e "y\n" | lvcreate -L 1G vg -n var+log
echo -e "y\n" | lvcreate -L 1G vg -n swap_1
# modify this to your definied filesystem
mkfs.ext4 /dev/mapper/vg-root
mkfs.ext4 /dev/mapper/vg-var+log
mkfs.ext4 /dev/mapper/vg-var
mkfs.ext4 /dev/mapper/vg-tmp
mkswap /dev/mapper/vg-swap_1

To get the script to the target machine, add a preseed early command that downloads the script and adds it as a finish hook. This will make partman set up the disk as it was supposed to, but after it is finished, the real partitioning is done with the script. Sadly, the preseed partitioning needs to be done, so partman validates correctly and activates the correct partitions so the install process can continue.

This is the partman configuration that I have successfully used.

d-i partman-auto/disk string /dev/sdb
d-i partman/alignment string optimal
d-i partman/confirm_nooverwrite boolean true
d-i partman-auto/method string lvm
d-i partman-lvm/device_remove_lvm boolean true
d-i partman-lvm/confirm boolean true
d-i partman-lvm/confirm_nooverwrite boolean true
d-i partman-auto-lvm/no_boot boolean true
d-i partman-auto-lvm/new_vg_name string vg
d-i partman-basicfilesystems/no_swap boolean false
d-i partman/early_command string \
wget http://<your-pxe-server>/hack.sh -O /lib/partman/finish.d/01cleanup;\
chmod +x /lib/partman/finish.d/01cleanup

# your disk configuration goes here

d-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true

d-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean false
d-i grub-installer/choose_bootdev string /dev/sda

Let me know if you have found a better way to do a two disk layout with partman, as a partitionless system disk really speeds up the daily work as an admin.

social