A Linux NAS setup with software RAID and incremental backup

Characteristics

I wanted to make my own Network Attached Storage (NAS) device with the following characteristics:

1. It should have lots of disk space and new disk space should be easy to add.
2. The primary storage should be redundant: if one disk fails, the data should still be there.
3. Important data on the primary storage should be backed up automatically and incrementally to secondary storage.
4. It should be able to run VMware.

Hardware

The following hardware was used to construct the NAS:

• Antec NSK4480 mini tower with 380W power supply
• Asus M3N78-CM GeForce 8200 SATA2 mainboard
• AMD Athlon X2 5050e 2.6GHz 1MB 45W AM2 processor
• Scythe Ninja Mini processor cooler
• Kingston 2x2GB DDR2 SDRAM PC6400 memory
• One Samsung HD501LJ 500GB disk
• Three Western Digital WD10EACS 1TB disks
  
• Samsung SH-S223F DVD player

The 500GB disk will be used to hold the operating system, Ubuntu 8.10 desktop, and the backup of the most important files. The three 1TB disks will be bundled together into a software RAID5 set, as shown in the following diagram:

Software RAID

We start by installing the operating system, Ubuntu 8.10 desktop for i386 systems. I choose the desktop version because I wanted to be able to access the VMware GUI on the NAS itself. I started with the 64 bit version of Ubuntu 8.10, but it was harder to install and it seemed slower than the i386 version. It does mean that we loose some of the 4GB RAM, but that is not a big concern for me.

After installing the OS, we move on to the partitioning of the three disks that will form the RAID5 set. We do this by installing the program gparted which is ideal for partitioning disks from a GUI:

sudo su -
apt-get install gparted
gparted

In gparted, create a single partition on the three RAID5 disks.
Next, create the RAID5 set:

apt-get install mdadm
mdadm --create /dev/md0 --level=5 --chunk=128 --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sdd1

Create a filesystem on the RAID5 set we just created:

mkfs.ext3 -b 4096 -R stride=32 /dev/md0

Create a mountpoint for the RAID5 set:

mkdir /data

Add an fstab entry by editing /etc/fstab:

/dev/md0   /data   /ext3   defaults   0   0

Finally, mount the RAID5 set:

mount /data

Incremental backup

The most important files on the NAS need to be backed up to the disk that also holds the operating system. The program rshapsnot is ideal for this task. It will do fast incremental backups that are browsable on the filesystem. So there’s no need to use a special program to restore the backup. Just copy the files and you’re done.

Install rsnapshot:

apt-get install rsnapshot

Edit the rsnapshot configuration file /etc/rsnapshot.conf:

snapshot_root /backup/
cmd_cp /bin/cp
cmd_ssh /usr/bin/ssh
cmd_du /usr/bin/du
cmd_rsnapshot_diff /usr/bin/rsnapshot-diff
interval hourly 6
interval daily 7
interval weekly 4
interval monthly 12
backup /data/important_files/ localhost/

This configuration file tells rsnapshot some basic things like the whereabouts of some needed programs and which directories it should backup. In our case, the directory /data/important_files is backed up to the directory /backup. It also tells that there should be 6 “hourly backups”, 7 “daily backups”, 4 “weekly backups” and 12 “monthly backups”. The reason I use quotation marks here, is that you yourself have to setup the cron jobs to actually run rsnapshot at the time you want. Run the following command:

crontab -e

and add the following lines:

# m h  dom mon dow   command
0 */4 * * * /usr/bin/rsnapshot hourly
45 23 * * * /usr/bin/rsnapshot daily
30 23 * * 0 /usr/bin/rsnapshot weekly
15 23 1 * * /usr/bin/rsnapshot monthly

This crontab will run the different backups in the following way:

• The “hourly backup” is run every four hours (denoted by the hour value of */4) on the hour (denoted by the minute value of 0).
• Every day at 23:45, the “daily backup” is run.
• Every sunday (denoted by the day-of-the-week value of 0) at 23:30 the “weekly backup” is run.
• Every first day of the month (denoted by the day-of-the-month value of 1) at 23:15 the “monthly backup” is run.

In combination with the “interval hourly 6” line in the rsnapshot configuration file, this means that the last 24 hours are covered with 6 backups. In the same way we can see that 7 days are covered in the last week, 4 weeks are covered in the last month and 12 months are covered in the last year.

File sharing

To actually start using the NAS, we will now install the Samba package:

apt-get install samba

Finally, we edit the configuration file /etc/samba/smb.conf:

security = user
[data]
   comment = Data
   browseable = yes
   path = /data
   guest ok = no
   read only = no
[backup]
   comment = Backup
   browseable = yes
   path = /backup
   guest ok = no
   read only = yes

This will make the primary storage accessible read-write through nasdata and the backup accessible read-only through nasbackup.