Installing Debian Raid

Instructions for installing a very clean Debian GNU/Linux system that boots from RAID 1, and has RAID 1 or RAID 5 root and data filesystems.

The examples assume two identical harddrives, sda and sdb, on which after a small boot partition, 1 GB is used for swap, 25 GB is used for the root filesystem and everything else is for a big “data” partition that will hold non-system stuff.

Although I personally prefer /boot to be readonly, this guide doesn’t add the ro flag in /etc/fstab, because that’ll only lead to complaints about lilo upgrades not going smoothly. (Which is exactly the point of having it readonly in the first place…)

They also assume some specific tools that you may or may not like, and a kernel without module support. This is how I prefer to do things for servers. Please don’t try to persuade me to use kernel packages, grub, modules, or whatever.

General knowledge is required. Don’t begin unless you understand each step.

Raidthingy

The 42nd time you do all this, it gets rather boring. So I decided to automate steps 4..19, excluding 15 (kernel), using a simple Perl script.

This script assumes you have equal and empty (unpartitioned) drives, and provides very little flexibility.

wget http://juerd.nl/files/raidthingy.pl
vim raidthingy.pl # Learn what it does.
perl raidthingy.pl # Follow instructions

The guide

1. Get a bootable cd with debootstrap and RAID support, like KNOPPIX, and boot from it.

2. Find a root shell.

If your drives are hdx instead of sdx, ensure that DMA is enabled for both drives:

hdparm -d1 /dev/hda
hdparm -d1 /dev/hdc

The rest of this guide assumes sda and sdb, because S-ATA devices are presented as SCSI devices in recent kernels.

3. Get networking up and running, if you don’t have DHCP:

ifconfig eth0 123.123.123.123 up
route add default gw 123.123.123.1
echo nameserver 123.123.123.1 > /etc/resolv.conf

4. Partition the drives, paying no attention to partition types yet:

fdisk /dev/sda
# n <CR> p <CR> 1 <CR> <CR> +64M <CR>
# n <CR> p <CR> 2 <CR> <CR> +1G <CR>
# n <CR> e <CR> 3 <CR> <CR> <CR>
# n <CR> l <CR> <CR> +25G <CR>
# n <CR> l <CR> <CR> <CR>
# a <CR> 1 <CR> w <CR>
fdisk /dev/sdb
# n <CR> p <CR> 1 <CR> <CR> +64M <CR>
# n <CR> p <CR> 2 <CR> <CR> +1G <CR>
# n <CR> e <CR> 3 <CR> <CR> <CR>
# n <CR> l <CR> <CR> +25G <CR>
# n <CR> l <CR> <CR> <CR>
# a <CR> 1 <CR> w <CR>

5. For every partition, create a RAID 1 array:

mdadm –create /dev/md0 -n 2 -l 1 /dev/sda1 /dev/sdb1
mdadm –create /dev/md1 -n 2 -l 1 /dev/sda2 /dev/sdb2
mdadm –create /dev/md2 -n 2 -l 1 /dev/sda5 /dev/sdb5
mdadm –create /dev/md3 -n 2 -l 1 /dev/sda6 /dev/sdb6

If you plan on installing sdb later, use missing instead of the second device.

For RAID 5, use -l 5 instead of -l 1, and just specify more disks. Make sure the boot volume is RAID 1, not 5 (/dev/md0 here, and yes, you can have RAID 1 with more than 2 disks). Also, increase the number of disks (-n) accordingly.

6. Create filesystems and initialize swap space:

mkfs.ext3 /dev/md0
mkswap /dev/md1
mkfs.ext3 /dev/md2
mkfs.ext3 /dev/md3 -O dir_index

7. Create a target mountpoint and mount your new filesystem(s) there:

mkdir /target
mount /dev/md2 /target

mkdir /target/boot
mount /dev/md0 /target/boot

mkdir /target/data
mount /dev/md3 /target/data

mkdir /target/data/home
mkdir /target/home
mount –bind /target/data/home /target/home

mkdir /target/data/var
mkdir /target/var
mount –bind /target/data/var /target/var

mkdir /data/share # for samba shares
mkdir /data/www # for mod_vhost_alias sites

8. Install a basic Debian system using debootstrap:

debootstrap sarge /target http://ftp.nl.debian.org/debian

9. Delete symlinks to the outside world in the target’s etc:

cd /target/etc
rm hostname resolv.conf localtime

10. Get some real things there:

cp /etc/resolv.conf .
ln -s /usr/share/zoneinfo/Europe/Amsterdam localtime
echo newboxthingy > hostname
vim default/rcS # FSCKFIX=yes

11. Set up the file system table:

# This is /etc/fstab
/dev/md0 /boot ext3 defaults 0 1
/dev/md1 none swap swap
/dev/md2 / ext3 defaults,errors=remount-ro 0 1
/dev/md3 /data ext3 defaults,errors=remount-ro 0 1
/data/home /home bind bind
/data/var /var bind bind
proc /proc proc

12. Get a working sources.list:

cd apt
rm sources.list
wget http://juerd.nl/sources.list

13. Change the current root directory to enter the new system:

chroot /target

14. Get the system up to date and install some useful packages:

apt-get update
apt-get dist-upgrade
apt-get install less wget w3m vim libncurses5-dev make gcc
mbr bzip2 lilo mdadm ssh

When asked to start RAID arrays automatically, answer “No”, as this is not needed with RAID built into the kernel and autodetected arrays.

15. Download a kernel, configure it, compile it, copy it:

cd /usr/src
wget http://ftp.nl.kernel.org/pub/linux/kernel/v2.6/linux-2.6.12.tar.bz2
tar -jvxf linux-2.6.12.tar.bz2
ln -s linux-2.6.12 linux

# grsecurity (optional)
wget http://www.grsecurity.net/grsecurity-2.1.6-2.6.11.12-200506141713.patch.gz
cd linux
zcat ../grsecurity-2.1.6-2.6.11.12-200506141713.patch.gz | patch -p1

cd /usr/src/linux
make menuconfig
# Don’t forget to compile in RAID 1/5 and ext3 support.
make bzImage
cp arch/i386/boot/bzImage /boot/vmlinuz-2.6.11.9
cp System.map /boot/System.map-2.6.11.9

16. Configure LILO:

# This is /etc/lilo.conf
boot=/dev/md0
root=/dev/md2
compact
lba32
read-only
image=/boot/vmlinuz-2.6.11.9
label=Linux

17. Install the boot records:

lilo
install-mbr /dev/sda
install-mbr /dev/sdb

18. Configure networking:

# This is /etc/network/interfaces
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 123.123.123.123
netmask 255.255.255.0
gateway 123.123.123.1
network 123.123.123.0
broadcast 123.123.123.255
# Alternatively:
# iface eth0 inet dhcp

19. Secure things a little:

shadowconfig on
passwd
adduser foo

20. Exit the chrooted environment:

exit

21. Wait until synchronization is complete:

watch cat /proc/mdstat

22. Set the partition types to 0xFD:

fdisk /dev/sda
# t <CR> 1 <CR> fd <CR>
# t <CR> 2 <CR> fd <CR>
# t <CR> 5 <CR> fd <CR>
# t <CR> 6 <CR> fd <CR> w <CR>
fdisk /dev/sdb
# t <CR> 1 <CR> fd <CR>
# t <CR> 2 <CR> fd <CR>
# t <CR> 5 <CR> fd <CR>
# t <CR> 6 <CR> fd <CR> w <CR>

23. Reboot and bring your favourite god(s) sacrifices:

reboot

24. Consider donating:

• Software in the Public Interest, Inc. (Debian)
• Free Software Foundation (GNU)
• Apache Software Foundation
• Perl Development Fund