GeodSoft logo   GeodSoft

Dual and Multi Booting FreeBSD, Linux, and OpenBSD

Booting Three and More Systems from One Disk
Triple Boot Systems

After different combinations of dual boot systems, it seems natural that a triple boot system containing one copy each of FreeBSD, Linux, and OpenBSD would be of the most interest. Hopefully what's already been said would provide sufficient information that most readers could build such a system at this point. Though I've put together several systems with three to ten independent booting systems, each including one or more copy each of FreeBSD, Linux, and OpenBSD, it wasn't until I consistently wiped the disks clean before starting, and doing OpenBSD first, Linux second, and FreeBSD last that I eliminated all install irregularities except a few minor warning messages.

Below are step by step instructions for a triple boot system that had no errors and required no workarounds. There were a few minor warnings from both OpenBSD and Linux.

There are two good reasons for having a triple boot machine. As a secondary machine, such a system makes a good learning environment for all three systems at the minimum cost and use of space. Attempting to add three different operating systems to an existing primary system just isn't a plausible option.

A triple boot machine with FreeBSD, Linux, and OpenBSD makes an excellent evaluation machine for comparing the operating systems themselves or any products that run on them. The best solution may be three identical machines but this is also the most expensive and space consuming solution. Over any length of time it's very hard to keep three systems identical in terms of hardware. It's not easy to be sure even new machines are identical as the same brand and model may not have identical components, and even if the components are identical they may not have been installed in the same locations or with the same settings.

Perhaps the best solution to evaluating three systems is a single machine with three identical drives in removable trays. This is still likely to be a moderately expensive solution unless two spare drives of the same make and model as one already installed just happen to being laying around. As a practical matter, this will likely mean buying three new drives and trays at the same time. Hard disks of the same make and model and similar manufacture dates are likely to be about as similar as any two computer components can be. Using them in the same system, ensures that all other components are in fact identical and installed and configured identically. In some ways three identical removable drives makes the best comparison environment. The only drawback compared to three separate systems is that side by side comparisons cannot be made.

A multi boot drive is almost as good as removable drives and much less expensive. There are two issues to consider. The boot process itself is of course not the same as the operating systems installed natively. This could be of some significance in operating system comparisons. The other issue is disk performance. The inner tracks may not perform as well as the outer tracks and the difference could be significant. If disk performance is an important part of the evaluation, the disk should be tested.

As FreeBSD is by far the easiest of the three systems to install in a native multi boot configuration, I'd use FreeBSD to evaluate the inner and outer disk tracks. I'd divide the disk into three equal parts and do three identical installations that include several subpartitions. I'd then find some disk benchmarks as well as run the applications to be evaluated on all three systems. The intent would be to determine if there is a performance difference, and if so is it significant and consistent. In other words, if there is a difference is it practical to establish a valid weighting method that would adjust for any measured differences.

If the application, as it will actually be used, consistently performs faster on the outer tracks, given identical OS and application installs, that's a pretty good indication how much the disk performance should be weighted. If the outer system performs 10% faster than the middle and 20% faster than the inner, then those amounts should be reasonable factors by which to adjust disk intensive performance figures when different OSs are installed in each disk area. One could go so far as to perform all performance tests on all three copies of FreeBSD and then on each of the three systems. The two sets of FreeBSD numbers should be very close when the disk area is the same or something is suspect in the testing procedure. Any additional performance variations are likely to be OS related or related to how the specific application interacts with the OS.

Step by Step Triple Boot

Start with a clean disk. Because this takes so long (over an hour), most of my installs started with less than completely clean disks. It's hard to know how much of the odd behaviour I saw from OpenBSD's fdisk and disklabel and Linux's fdisk was caused by the immediately preceding installs or by old disk information that was not overwritten by subsequent installs. When I started with clean disks and did OpenBSD then Linux then FreeBSD, all odd behavior disappeared.

Install OpenBSD first into the first slice. Use OpenBSD's fdisk to create only it's own partition and use 0. Enter 'e 0' then 'a6' as the partition type. When asked if you wish to edit in CHS mode, change the default 'n' to 'y' and press enter. Enter cylinder 0, head 1 (not 0), and sector 1 for the cylinder start. For the ending cylinder divide the ending cylinder number displayed just before the colon in the prompt by 3 and round the result. If you do not plan an equal space allocation, then compute the ending cylinder from the percent of the total cylinders based on the space you want to allocate to OpenBSD. Enter the ending head as the last head available and the ending sector as the last sector available. Mark the first partition bootable with "f 0" then write your changes ('w') and quit ('q') which will take you into disklabel. When you write the change you should see the "no disk label" message.

When OpenBSD installs first, into the first cylinder and slice, on a clean disk, disklabel should behave normally and properly calculate starting offsets and entered relative sizes, where numbers ending in 'm' indicate megabytes and numbers ending with a 'g' indicate gigabytes. Allocate your available space into the devices and mount points as you would normally. The rest of the OpenBSD install is identical to any other.

Follow OpenBSD with a Linux install. Use the fdisk (expert) method. You should not see any errors or warnings when entering fdisk. Print the existing partition information. You should see primary partition 1, recognized as an OpenBSD partition with starting and ending cylinder numbers one higher than you entered in OpenBSD's fdisk. Linux counts cylinders from 1. Create primary partition 2 using the default starting cylinder and enter the ending cylinder or the number of megabytes ("+999m") or gigabytes ("+9g") that you wish to allocate to FreeBSD. Use 't' to change the partition type for the second partition to 'a5'.

Create partition 4 as an extended partition leaving the third slice unused. You could use it for a Linux primary /boot or / partition but that's a waste. A latter reinstall of FreeBSD might make much better use of an extra slice than Linux, which can only put a single file system and mount point in a primary partition. In the extended partition create hda5 for '/' and hda6 for Linux swap. Allocate the rest of the space in the extended partition as you would normally for a Linux system.

Use 'p' frequently to display your partitions. When you have 1 as a6 OpenBSD, 2 as a5 BSD/386 and 4 as 5 extended with hda5 - hda6-12 allocated with appropriate amounts of space, save your changes and exit with 'w'.

Use the edit option only, in Disk Druid to assign your mount points and format types for hda5 to hda6-12 including the swap partition. Install GRUB into the boot partition not the default MBR. The rest of the install except the time setting is like any other Linux install. Check the box that indicates the system clock uses UTC time or the Linux time won't match the OpenBSD time.

Be sure to create a boot floppy. With '/' mounted in hda5 and GRUB also installed in hda5, you must create a boot floppy to boot the system the first time. Linux boot floppies are root device specific. If you make a habit of always installing Linux in the extended partition and making hda5 your boot device, you can share boot floppies. If this is the first time you've installed Linux this way make the floppy and label it. If you have other Linux systems and can mount a boot floppy on one of them, changing the last line in syslinux.cfg where "root=boot device" to /dev/hda5 will make it useable on the new system.

As soon as you've finished installing Linux, boot it from the floppy. cd to /boot/grub. edit grub.conf and add the following lines.

title OpenBSD 3.0
        root (hd0,0)
        makeactive
        chainloader +1
title FreeBSD 4.4 
        root (hd0,1,a)
        kernel /boot/loader

These will match the slices described above. After saving your changes to grub.conf, start GRUB interactively to make the hard disk bootable with grub. At the GRUB prompt enter the following:

GRUB> root (hd0,4)
GRUB> setup (hd0)
GRUB> quit

When you boot the system next, you should see the GRUB menu and OpenBSD and Linux should be bootable but FreeBSD still needs to be installed.

Start the FreeBSD install and in fdisk, verify that the partitions match those created by OpenBSD and Linux's fdisk. Make no changes and use 'q' to quit and continue. At the boot manager prompt, change the default to none. In disklabel, assign the partitions as you would normally for the amount of space available. Remember to change the time default so FreeBSD knows the system (CMOS) clock uses UTC. The rest of the FreeBSD is completely standard; use whatever choices you would on any other install.

When you reboot after the FreeBSD install you should have three bootable Unix like systems installed, selected by GRUB at boot time, and defaulted to boot Linux after 10 seconds unless you make another choice first.

Many Boot Systems

When I started this project, I just wanted a way to limit what I was spending on extra hard disks for test and development projects, while huge amounts of space went unused on oversized drives. I wasn't even sure if it was practical to make one copy each of FreeBSD, Linux, and OpenBSD work together on a single disk. While there is plenty of literature about dual booting Windows systems, the literature on dual booting two or more open source systems is limited. After I'd dual booted each of the different combinations, the extraordinary luck I had triple booting FreeBSD, Linux, and OpenBSD successfully the first time I tried, was a major impetus to learn more. After that, I had to know what the limits really were.

My first quad boot attempt showed me how lucky I'd been with the triple boot machine. The first quad took 27 different OS installs or attempts and required wiping the disk clean 4 times using Ranish Partition Manager and even then the importance of starting with a clean disk wasn't yet apparent. By the time the first quad boot was working, I thought I knew the gotchas, but it took multiple quad, quint and up to ten system installs before I really understood what the limits were and what was causing problems with which programs.

The biggest breakthrough was booting two OpenBSD systems on the same hard disk, with two other bootable systems. The final important steps were booting multiple FreeBSD systems in one slice and the realization that Disk Druid would install single '/' file system Linux systems in any available partition and that despite the error messages suggesting a limit of 8, fdisk could allocate 12 partitions inside an extended partition.

This provided the basis to determine upper limits for each system on one disk as well as how many different systems could be installed in any combination. If a disk is partitioned appropriately at the begining, it can hold bootable systems in any mixture up to the limits imposed by disk space, the four slices per disk limit, and each system's intrinsic limits.

Repeating the numbers given previously, it is possible to install 4 OpenBSD, 14 Linux, or 27 FreeBSD bootable systems on a single disk. Further you can mix these to get: 2 OpenBSD, 6 FreeBSD and 11 Linux; or 1 OpenBSD, 11 Linux, and 13 FreeBSD; or 11 Linux and 20 FreeBSD, bootable systems on one hard disk. These limits come from the fact a disk can have a maximum of four slices. OpenBSD requires a slice per system. FreeBSD can make 7 partitions per slice, put a full system on a single partition, and requires one partition for swap, which can be shared by all FreeBSD systems on the disk. Linux can create 1 partition in each of 3 primary slices and 12 partitions in one extended partition, but also requires one swap partition that can be used by any Linux system on the disk. Using GRUB to hide extended partitions like I've used it to hide OpenBSD partitions, it should be possible to create 4 extended partitions, where normally only one is allowed, and install up to 44 Linux systems on a disk; I haven't and don't plan to test this.

What is the most cost effective way to make use of space for FreeBSD, Linux, and OpenBSD test and development projects? Since OpenBSD is still limited to booting from the first 1024 cylinders and requires an entire slice, this is the primary practical limit if a diverse mixture of systems is planned. As long as I've done Unix systems, I've carefully partitioned disks into multiple file systems based on anticipated use. This still makes sense for production machines, but for the kind system that I'm discussing, it's not important, and prevents making use of the large amounts of disk space available on contemporary drives.

The smallest high quality drive widely available today is 40GB with 38+GB usable disk space. This suggests 10 systems per drive. 4GB test systems are really quite generous since a "full" Red Hat Linux install is still under 1.7GB and a basic server install without X is about 665MB. An OpenBSD system, including development tools and system source code, is about 145MB and a minimum FreeBSD install is not much over 100MB but could grow larger than Red Hat if you got carried away installing extra packages.

With a target of 10 systems per drive, 3 - 3 - 4 isn't an option because of OpenBSD's limits. 2 - 4 - 4 makes sense though. Because the FreeBSD and Linux systems can respectively share their own swap partitions and a common large /tmp and each OpenBSD system must have these individually, this suggests OpenBSD systems of 6GB each. I know the 1024 cylinder limit is somewhere between 7 and 8GB without repeating the calculations so 6GB works well with that. The second system will still have a lot more than '/' inside the 1024 boundary. This leaves 26GB or 13GB each for FreeBSD and Linux, assuming you allocate space equally.

The systems that the drives will be used in have from 256 to 640MB RAM. For ease of calculation and a comfort zone, 1GB swap is generous; depending on what the systems do, it could probably be reduced to a fraction of that without noticeable impact.

This leaves 12GB for four systems. I like the idea of large /tmp file systems that can be shared by each group of systems. An even split would be 2.4GB each but if each system is reduced to 2.3GB this leaves for 2.8GB /tmp. Since each system will have access to the shared /tmp and swap, their effective sizes will be 2.3 + 2.8 + 1 = 6.1GB. In a way, a 40GB drive has ten 6GB systems with two OpenBSD, four FreeBSD, and Four Linux systems. I set up one 40GB drive just as this is described.

Six systems divided 2-2-2 on a 30GB drive with 28+GB usable space seems reasonable. There is no need to make the allocations equal. OpenBSD lends itself well to small server installs and Linux (at least Red Hat) is much more space hungry so the Linux systems might be made the largest, OpenBSD the smallest, and FreeBSD somewhere between. One or two more systems could easily fit. If more OpenBSD systems were wanted, smaller 10 - 20GB drives could handle 2 to 4 systems each, though the 1024 cylinder limit makes it difficult to divide the space intelligently.

The first many boot drive I set up was the 40GB drive described above. I started with Linux, and though I got the OpenBSD systems installed, there were significant issues doing so.

After this first many boot drive, I settled on wiping the disks clean and starting with OpenBSD installs. After two OpenBSD systems are set up, the first Linux system is installed and all Linux partitions created and an empty 'a5' slice reserved for FreeBSD. After that is done, it makes no difference in what order any remaining Linux and FreeBSD systems are set up. GRUB is the boot manager and set up to do other systems as soon as the first Linux install is completed.

A many boot drive is done just as the triple boot system described above but with an optional extra OpenBSD system done as described at the end of OpenBSD Installs. Instead of allocating additional partitions in the FreeBSD and Linux areas to the tradition mount points of /usr, /usr/local, /var, /home, /tmp, /opt, etc., they are used to do additional '/' file systems on which complete systems are installed. Some file systems such as /tmp or /home might be shared among multiple systems. As any FreeBSD can mount any device that contains a FreeBSD file system and any Linux system can do the same, if one or both OpenBSD systems are not in the mix there is a lot of latitude for experimenting with a substantial number of partitions that may be shared among like systems.

GRUB was placed in the boot partition for the first and every subsequent Linux install. The "None" option to make no MBR changes was selected for all FreeBSD installs rather than the default of installing the FreeBSD boot loader. Once GRUB is set up in the first Linux system, putting GRUB from any subsequent install in the MBR, installing any other boot loader, or making any changes to GRUB's IPL code, would render all the other GRUB booted systems as unbootable, until GRUB is reinstalled.

While only a very few are likely to have need of systems that contain more than three bootable systems on a drive, a few may find themselves in a situation similar to mine and hopefully this will have given you enough information to set up the systems you need. Some of the uses I'll put many boot systems to use for in the future follow. With each OpenBSD release, I've built custom kernels. Kernels build most reliably on basic systems immediately following a standard install. By the time I've put a new system into production use, I've removed the ability build a new kernel. Having many bootable systems will let me keep an OpenBSD system just for kernel building. I've built and tested various firewall configurations on both OpenBSD and Linux including both bridging and standard routing with NAT. After I finished the project that used these, I archived these systems because I needed the disks. Many boot systems would let me keep these available to experiment with other small networks when the need arises. I once built an "attack" machine with nmap and every network probing and analysis tool I could find. This is great for testing a new firewall or system setup but for obvious reasons you don't want it left online all the time. This also got archived. Recently I've started building custom Apache servers with additional modules. When I put these web servers to use in a production environment, I won't want the development tools on the production server. Many boot systems will let me keep a development machine available (but offline) for whenever the next custom server is needed due to a new needed feature or a new version released for security reasons. Now that I know how to build them reliably, I expect many boot systems to be an important part of my future disk use.

transparent spacer

Top of Page - Site Map

Copyright © 2000 - 2014 by George Shaffer. This material may be distributed only subject to the terms and conditions set forth in http://GeodSoft.com/terms.htm (or http://GeodSoft.com/cgi-bin/terms.pl). These terms are subject to change. Distribution is subject to the current terms, or at the choice of the distributor, those in an earlier, digitally signed electronic copy of http://GeodSoft.com/terms.htm (or cgi-bin/terms.pl) from the time of the distribution. Distribution of substantively modified versions of GeodSoft content is prohibited without the explicit written permission of George Shaffer. Distribution of the work or derivatives of the work, in whole or in part, for commercial purposes is prohibited unless prior written permission is obtained from George Shaffer. Distribution in accordance with these terms, for unrestricted and uncompensated public access, non profit, or internal company use is allowed.

 
Home >
How-To >
Dual Boot >
manyboot.htm


What's New
How-To
Opinion
Book
                                       
Email address

Copyright © 2000-2014, George Shaffer. Terms and Conditions of Use.