As few of you might know, I made my own SUSE-based LiveCDs a while ago, using (like Knoppix) cloop compression with iso9660 and my own kernel code for file-based overlay to make it writeable. You might be amazed at how fast it runs in KVM. However, the kernel part has bit-rotten and there are other techniques out there today, so I took a look around at how others do their LiveCDs.

But first some broader overview. To make a LiveCD, the biggest problem is that CDs are not writeable (and even modern Flash devices do not want to be written too much). Embedded devices using flash had the same problem. Various approaches have been used in the past to solve this:

• adapt all software to write into ram-disks e.g. by having symlinks (hard to create and maintain)
• load all software into RAM (only for small distributions)
• use file-based overlaying such as unionfs or aufs to have software write into RAM (lsof, pwd, and hardlinks can be tricky)
• use block-based overlaying (problem: can not easily free disk space again)

Also compression is used to fit more onto a CD. And interestingly, this usually also speeds up booting because it is faster to read 10MB off a CD and decompress it into the original 30MB than to read 30MB from such a slow medium.

Now, to the distributions.

• openSUSE has the classic DVD installs that use special installation-images and run in RAM and then there are the real LiveCDs that are created by our kiwi tool, use block-based overlaying and LZMA compression of a ext3 by means of our FUSE-based clicfs.
• All of the other distributions use squashfs for compression. Mageia employs dracut for initrd and unionfs for file-based overlaying
• Debian uses aufs for file-based overlaying
• Ubuntu uses overlayfs for file-based overlaying
• Fedora uses an ext4 filesystem image contained in a squashfs with dm-snapshot for block-based overlaying, thus being most similar to openSUSE

I also spent some time benchmarking (on my AMD A10-5800K) the various technologies with a simple script using Debians uncompressed rootfs of 495132 KiB as data.
squashfs supports three different compression methods: lzo, gzip and xz (aka LZMA).

• squashfs-lzo: size:220992 compression:11.1MB/s decompression:134.4MB/s
• squashfs-gzip: size:203328 compression:15.5MB/s decompression:88.9MB/s
• squashfs-xz: size:176064 compression:6.5MB/s decompression:22.5MB/s
• cloop(gzip): size:213348 compression:16.2MB/s decompression:49.6MB/s
• clicfs(xz): size:185300 compression:16.7MB/s decompression:18.2MB/s

This has some surprises: even when using the same compression method, sizes can differ by 5% and speed can differ even more.

If you want to compare numbers on your system, memory throughput is also interesting:
# dd if=/dev/zero of=/dev/null bs=1M count=100000
104857600000 bytes (105 GB) copied, 12.4499 s, 8.4 GB/s

Overall, clicfs is performing OK, considering that it already takes care of the overlaying, but for my own LiveCD I would prefer Debian’s method and I am wondering how it would work.