Dual and Multi Booting FreeBSD, Linux, and OpenBSD
Booting Three and More Systems from One Disk
Triple Boot Systems
After different combinations of dual boot systems, it seems natural
that a triple boot system containing one copy each of FreeBSD, Linux,
and OpenBSD would be of the most interest. Hopefully what's already
been said would provide sufficient information that most readers could
build such a system at this point. Though I've put together several
systems with three to ten independent booting systems, each including
one or more copy each of FreeBSD, Linux, and OpenBSD, it wasn't until
I consistently wiped the disks clean before starting, and doing
OpenBSD first, Linux second, and FreeBSD last that I eliminated all
install irregularities except a few minor warning messages.
Below are step by step instructions for a triple
boot system that had no errors and required no workarounds. There
were a few minor warnings from both OpenBSD and Linux.
There are two good reasons for having a triple boot machine. As a
secondary machine, such a system makes a good learning environment for
all three systems at the minimum cost and use of space. Attempting to
add three different operating systems to an existing primary system
just isn't a plausible option.
A triple boot machine with FreeBSD, Linux, and OpenBSD makes an
excellent evaluation machine for comparing the operating systems
themselves or any products that run on them. The best solution may be
three identical machines but this is also the most expensive and space
consuming solution. Over any length of time it's very hard to keep
three systems identical in terms of hardware. It's not easy to be
sure even new machines are identical as the same brand and model may
not have identical components, and even if the components are
identical they may not have been installed in the same locations or
with the same settings.
Perhaps the best solution to evaluating three systems is a single machine
with three identical drives in removable trays. This is still likely to
be a moderately expensive solution unless two spare drives of the same
make and model as one already installed just happen to being laying around.
As a practical matter, this will likely mean buying three new drives and
trays at the same time. Hard disks of the same make and model and similar
manufacture dates are likely to be about as similar as any two computer
components can be. Using them in the same system, ensures that all other
components are in fact identical and installed and configured identically.
In some ways three identical removable drives makes the best comparison
environment. The only drawback compared to three separate systems is that
side by side comparisons cannot be made.
A multi boot drive is almost as good as removable drives and much less
expensive. There are two issues to consider. The boot process itself
is of course not the same as the operating systems installed natively.
This could be of some significance in operating system comparisons.
The other issue is disk performance. The inner tracks may not perform
as well as the outer tracks and the difference could be significant.
If disk performance is an important part of the evaluation, the disk
should be tested.
As FreeBSD is by far the easiest of the three systems to install in a
native multi boot configuration, I'd use FreeBSD to evaluate the inner
and outer disk tracks. I'd divide the disk into three equal parts and
do three identical installations that include several subpartitions.
I'd then find some disk benchmarks as well as run the applications to
be evaluated on all three systems. The intent would be to determine if
there is a performance difference, and if so is it significant and
consistent. In other words, if there is a difference is it practical
to establish a valid weighting method that would adjust for any
measured differences.
If the application, as it will actually be used, consistently performs
faster on the outer tracks, given identical OS and application
installs, that's a pretty good indication how much the disk
performance should be weighted. If the outer system performs 10%
faster than the middle and 20% faster than the inner, then those
amounts should be reasonable factors by which to adjust disk intensive
performance figures when different OSs are installed in each disk
area. One could go so far as to perform all performance tests on all
three copies of FreeBSD and then on each of the three systems. The
two sets of FreeBSD numbers should be very close when the disk area is
the same or something is suspect in the testing procedure. Any
additional performance variations are likely to be OS related or
related to how the specific application interacts with the OS.
Step by Step Triple Boot
Start with a clean disk. Because this
takes so long (over an hour), most of my installs started with less
than completely clean disks. It's hard to know how much of the odd
behaviour I saw from OpenBSD's fdisk and disklabel and Linux's fdisk
was caused by the immediately preceding installs or by old disk
information that was not overwritten by subsequent installs. When I
started with clean disks and did OpenBSD then Linux then FreeBSD, all
odd behavior disappeared.
Install OpenBSD first into the first slice. Use OpenBSD's fdisk to
create only it's own partition and use 0. Enter 'e 0' then 'a6' as
the partition type. When asked if you wish to edit in CHS mode, change
the default 'n' to 'y' and press enter. Enter cylinder 0, head 1 (not
0), and sector 1 for the cylinder start. For the ending cylinder
divide the ending cylinder number displayed just before the colon in
the prompt by 3 and round the result. If you do not plan an equal
space allocation, then compute the ending cylinder from the percent of
the total cylinders based on the space you want to allocate to
OpenBSD. Enter the ending head as the last head available and the
ending sector as the last sector available. Mark the first partition
bootable with "f 0" then write your changes ('w') and quit ('q')
which will take you into disklabel. When you write the change you
should see the "no disk label" message.
When OpenBSD installs first, into the first cylinder and slice, on a
clean disk, disklabel should behave normally and properly calculate
starting offsets and entered relative sizes, where numbers ending in
'm' indicate megabytes and numbers ending with a 'g' indicate
gigabytes. Allocate your available space into the devices and mount
points as you would normally. The rest of the OpenBSD install is
identical to any other.
Follow OpenBSD with a Linux install. Use the fdisk (expert) method.
You should not see any errors or warnings when entering fdisk.
Print the existing partition information. You should see primary
partition 1, recognized as an OpenBSD partition with starting and ending
cylinder numbers one higher than you entered in OpenBSD's fdisk. Linux
counts cylinders from 1. Create primary partition 2 using the default
starting cylinder and enter the ending cylinder or the number of megabytes
("+999m") or gigabytes ("+9g") that you wish to allocate to FreeBSD.
Use 't' to change the partition type for the second partition to 'a5'.
Create partition 4 as an extended partition leaving the third slice
unused. You could use it for a Linux primary /boot or / partition but
that's a waste. A latter reinstall of FreeBSD might make much better
use of an extra slice than Linux, which can only put a single file
system and mount point in a primary partition. In the extended
partition create hda5 for '/' and hda6 for Linux swap. Allocate the
rest of the space in the extended partition as you would normally for
a Linux system.
Use 'p' frequently to display your partitions. When you have 1 as a6
OpenBSD, 2 as a5 BSD/386 and 4 as 5 extended with hda5 - hda6-12
allocated with appropriate amounts of space, save your changes and
exit with 'w'.
Use the edit option only, in Disk Druid to assign your mount points
and format types for hda5 to hda6-12 including the swap partition.
Install GRUB into the boot partition not the default MBR. The rest of
the install except the time setting is like any other Linux install.
Check the box that indicates the system clock uses UTC time or the
Linux time won't match the OpenBSD time.
Be sure to create a boot floppy. With '/' mounted in hda5 and GRUB
also installed in hda5, you must create a boot floppy to boot the
system the first time. Linux boot floppies are root device specific.
If you make a habit of always installing Linux in the extended
partition and making hda5 your boot device, you can share boot
floppies. If this is the first time you've installed Linux this way
make the floppy and label it. If you have other Linux systems and can
mount a boot floppy on one of them, changing the last line in
syslinux.cfg where "root=boot device" to /dev/hda5 will make it
useable on the new system.
As soon as you've finished installing Linux, boot it from the floppy.
cd to /boot/grub. edit grub.conf and add the following lines.
title OpenBSD 3.0
root (hd0,0)
makeactive
chainloader +1
title FreeBSD 4.4
root (hd0,1,a)
kernel /boot/loader
These will match the slices described above. After saving your
changes to grub.conf, start GRUB interactively to make the hard disk
bootable with grub. At the GRUB prompt enter the following:
GRUB> root (hd0,4)
GRUB> setup (hd0)
GRUB> quit
When you boot the system next, you should see the GRUB menu and OpenBSD
and Linux should be bootable but FreeBSD still needs to be installed.
Start the FreeBSD install and in fdisk, verify that the partitions
match those created by OpenBSD and Linux's fdisk. Make no changes and
use 'q' to quit and continue. At the boot manager prompt, change the
default to none. In disklabel, assign the partitions as you would
normally for the amount of space available. Remember to change the
time default so FreeBSD knows the system (CMOS) clock uses UTC.
The rest of the FreeBSD is completely standard; use whatever choices
you would on any other install.
When you reboot after the FreeBSD install you should have three bootable
Unix like systems installed, selected by GRUB at boot time, and defaulted
to boot Linux after 10 seconds unless you make another choice first.
Many Boot Systems
When I started this project, I just wanted a way to limit what I was
spending on extra hard disks for test and development projects, while
huge amounts of space went unused on oversized drives. I wasn't even
sure if it was practical to make one copy each of FreeBSD, Linux, and
OpenBSD work together on a single disk. While there is plenty of
literature about dual booting Windows systems, the literature on dual
booting two or more open source systems is limited. After I'd dual
booted each of the different combinations, the extraordinary luck I
had triple booting FreeBSD, Linux, and OpenBSD successfully the first
time I tried, was a major impetus to learn more. After that, I had to
know what the limits really were.
My first quad boot attempt showed me how lucky I'd been with the
triple boot machine. The first quad took 27 different OS installs or
attempts and required wiping the disk clean 4 times using Ranish
Partition Manager and even then the importance of starting with a
clean disk wasn't yet apparent. By the time the first quad boot was
working, I thought I knew the gotchas, but it took multiple quad,
quint and up to ten system installs before I really understood what
the limits were and what was causing problems with which programs.
The biggest breakthrough was booting two
OpenBSD systems on the same hard disk, with two other bootable
systems. The final important steps were booting multiple FreeBSD
systems in one slice and the realization that Disk Druid would install
single '/' file system Linux systems in any available partition and
that despite the error messages suggesting a limit of 8, fdisk could
allocate 12 partitions inside an extended partition.
This provided the basis to determine upper limits for each system on
one disk as well as how many different systems could be installed in
any combination. If a disk is partitioned appropriately at the
begining, it can hold bootable systems in any mixture up to the limits
imposed by disk space, the four slices per disk limit, and each
system's intrinsic limits.
Repeating the numbers given previously, it is possible to install 4
OpenBSD, 14 Linux, or 27 FreeBSD bootable systems on a single disk.
Further you can mix these to get: 2 OpenBSD, 6 FreeBSD and 11 Linux;
or 1 OpenBSD, 11 Linux, and 13 FreeBSD; or 11 Linux and 20 FreeBSD,
bootable systems on one hard disk. These limits come from the fact a
disk can have a maximum of four slices. OpenBSD requires a slice per
system. FreeBSD can make 7 partitions per slice, put a full system on
a single partition, and requires one partition for swap, which can be
shared by all FreeBSD systems on the disk. Linux can create 1
partition in each of 3 primary slices and 12 partitions in one
extended partition, but also requires one swap partition that can be
used by any Linux system on the disk. Using GRUB to hide extended
partitions like I've used it to hide OpenBSD partitions, it should be
possible to create 4 extended partitions, where normally only one is
allowed, and install up to 44 Linux systems on a disk; I haven't and
don't plan to test this.
What is the most cost effective way to make use of space for FreeBSD,
Linux, and OpenBSD test and development projects? Since OpenBSD is
still limited to booting from the first 1024 cylinders and requires an
entire slice, this is the primary practical limit if a diverse mixture
of systems is planned. As long as I've done Unix systems, I've
carefully partitioned disks into multiple file systems based on
anticipated use. This still makes sense for production machines, but
for the kind system that I'm discussing, it's not important, and
prevents making use of the large amounts of disk space available on
contemporary drives.
The smallest high quality drive widely available today is 40GB with
38+GB usable disk space. This suggests 10 systems per drive. 4GB test
systems are really quite generous since a "full" Red Hat Linux install
is still under 1.7GB and a basic server install without X is about
665MB. An OpenBSD system, including development tools and system
source code, is about 145MB and a minimum FreeBSD install is not much
over 100MB but could grow larger than Red Hat if you got carried away
installing extra packages.
With a target of 10 systems per drive, 3 - 3 - 4 isn't an option
because of OpenBSD's limits. 2 - 4 - 4 makes sense though. Because
the FreeBSD and Linux systems can respectively share their own swap
partitions and a common large /tmp and each OpenBSD system must have
these individually, this suggests OpenBSD systems of 6GB each. I know
the 1024 cylinder limit is somewhere between 7 and 8GB without
repeating the calculations so 6GB works well with that. The second
system will still have a lot more than '/' inside the 1024 boundary.
This leaves 26GB or 13GB each for FreeBSD and Linux, assuming you
allocate space equally.
The systems that the drives will be used in have from 256 to 640MB
RAM. For ease of calculation and a comfort zone, 1GB swap is
generous; depending on what the systems do, it could probably be
reduced to a fraction of that without noticeable impact.
This leaves 12GB for four systems. I like the idea of large /tmp file
systems that can be shared by each group of systems. An even split
would be 2.4GB each but if each system is reduced to 2.3GB this leaves
for 2.8GB /tmp. Since each system will have access to the shared /tmp
and swap, their effective sizes will be 2.3 + 2.8 + 1 = 6.1GB. In a
way, a 40GB drive has ten 6GB systems with two OpenBSD, four FreeBSD,
and Four Linux systems. I set up one 40GB drive just as this is
described.
Six systems divided 2-2-2 on a 30GB drive with 28+GB usable space
seems reasonable. There is no need to make the allocations equal.
OpenBSD lends itself well to small server installs and Linux (at least
Red Hat) is much more space hungry so the Linux systems might be made
the largest, OpenBSD the smallest, and FreeBSD somewhere between. One
or two more systems could easily fit. If more OpenBSD systems were
wanted, smaller 10 - 20GB drives could handle 2 to 4 systems each,
though the 1024 cylinder limit makes it difficult to divide the space
intelligently.
The first many boot drive I set up was the 40GB drive described above.
I started with Linux, and though I got the OpenBSD systems installed,
there were significant issues doing so.
After this first many boot drive, I settled on wiping the disks clean
and starting with OpenBSD installs. After two OpenBSD systems are set
up, the first Linux system is installed and all Linux partitions
created and an empty 'a5' slice reserved for FreeBSD. After that is
done, it makes no difference in what order any remaining Linux and
FreeBSD systems are set up. GRUB is the boot manager and set up to do
other systems as soon as the first Linux install is completed.
A many boot drive is done just as the triple boot system described
above but with an optional extra OpenBSD system done as described at
the end of OpenBSD Installs. Instead
of allocating additional partitions in the FreeBSD and Linux areas to
the tradition mount points of /usr, /usr/local, /var, /home, /tmp,
/opt, etc., they are used to do additional '/' file systems on which
complete systems are installed. Some file systems such as /tmp or
/home might be shared among multiple systems. As any FreeBSD can
mount any device that contains a FreeBSD file system and any Linux
system can do the same, if one or both OpenBSD systems are not in the
mix there is a lot of latitude for experimenting with a substantial
number of partitions that may be shared among like systems.
GRUB was placed in the boot partition for the first and every
subsequent Linux install. The "None" option to make no MBR changes was
selected for all FreeBSD installs rather than the default of
installing the FreeBSD boot loader. Once GRUB is set up in the first
Linux system, putting GRUB from any subsequent install in the MBR,
installing any other boot loader, or making any changes to GRUB's IPL
code, would render all the other GRUB booted systems as unbootable,
until GRUB is reinstalled.
While only a very few are likely to have need of systems that contain
more than three bootable systems on a drive, a few may find themselves
in a situation similar to mine and hopefully this will have given you
enough information to set up the systems you need. Some of the uses
I'll put many boot systems to use for in the future follow. With each
OpenBSD release, I've built custom kernels. Kernels build most
reliably on basic systems immediately following a standard install. By
the time I've put a new system into production use, I've removed the
ability build a new kernel. Having many bootable systems will let me
keep an OpenBSD system just for kernel building. I've built and
tested various firewall configurations on both OpenBSD and Linux
including both bridging and standard routing with NAT. After I
finished the project that used these, I archived these systems because
I needed the disks. Many boot systems would let me keep these
available to experiment with other small networks when the need
arises. I once built an "attack" machine with nmap and every network
probing and analysis tool I could find. This is great for testing a
new firewall or system setup but for obvious reasons you don't want it
left online all the time. This also got archived. Recently I've
started building custom Apache servers with additional modules. When
I put these web servers to use in a production environment, I won't
want the development tools on the production server. Many boot
systems will let me keep a development machine available (but offline)
for whenever the next custom server is needed due to a new needed
feature or a new version released for security reasons. Now that I
know how to build them reliably, I expect many boot systems to be an
important part of my future disk use.
Top of Page -
Site Map
Copyright © 2000 - 2014 by George Shaffer. This material may be
distributed only subject to the terms and conditions set forth in
http://GeodSoft.com/terms.htm
(or http://GeodSoft.com/cgi-bin/terms.pl).
These terms are subject to change. Distribution is subject to
the current terms, or at the choice of the distributor, those
in an earlier, digitally signed electronic copy of
http://GeodSoft.com/terms.htm (or cgi-bin/terms.pl) from the
time of the distribution. Distribution of substantively modified
versions of GeodSoft content is prohibited without the explicit written
permission of George Shaffer. Distribution of the work or derivatives
of the work, in whole or in part, for commercial purposes is prohibited
unless prior written permission is obtained from George Shaffer.
Distribution in accordance with these terms, for unrestricted and
uncompensated public access, non profit, or internal company use is
allowed.
|