Dual and Multi Booting FreeBSD, Linux, and OpenBSD with GNU Grub

Contents and Basics

• Dual Boot Pros and Cons
• Partition Defined
• Partitioning Basics
• The 1024 Cylinder Limit
• Combining Operating Systems
• FreeBSD and OpenBSD Only
• Linux with FreeBSD and or OpenBSD
• Linux Installs
• FreeBSD Installs
• OpenBSD Installs
• OpenBSD Partitioning Issues
• GNU GRUB Boot Loader
• Triple Boot Systems
• Step By Step
• Many Boot Systems

An in-depth tutorial on how-to install and boot two or more open source operating systems, specifically FreeBSD, Linux and or OpenBSD from the same hard disk. Booting from multiple hard disks in the same system will also be discussed briefly. The largely historical 1024 cylinder boot limit is covered. The emphasis is on step by step mechanics and not theory or background but some background is essential. How to boot any combination of two systems, as well as how to boot all three or more systems from a single disk is covered. All the experiments for these pages were done with the current releases, FreeBSD 4.4, Red Hat Linux 7.2 (and 7.1), and OpenBSD 3.0. Only Intel architecture dual or multi booting is discussed.

The current releases of FreeBSD and Linux do not have the 1024 cylinder boot limit nor do recent BIOSs. With powerful boot loaders like GNU GRUB, dual and multi booting systems is simple. There are some problems that affect specific situations. Hopefully these web pages identify most of these. The smallest hard disks available today have room for several installs of FreeBSD, Linux, and or OpenBSD.

Throughout this discussion, Linux means Red Hat 7.2 Linux. Much will be applicable to older Red Hat versions and other Linux distributions but the details will vary.

GNU GRUB, http://www.gnu.org/software/grub/, which in Red Hat 7.2, has replaced Lilo as the default boot loader, is used with all Linux installs and any install that includes more than one copy of OpenBSD or multiple installs of FreeBSD in a single slice. GRUB is a much more powerful boot loader than Lilo. GRUB will allow booting from virtually any partition on any hard disk or floppy. GRUB is operating system independent and can be compiled and installed from FreeBSD or OpenBSD and other systems.

To take things to the extreme, with the assistance of GRUB, it is possible to install 4 OpenBSD, 14 Linux, or 27 FreeBSD bootable systems on a single disk. Further you can mix these to get: 2 OpenBSD, 6 FreeBSD and 11 Linux; or 1 OpenBSD, 11 Linux, and 13 FreeBSD; or 11 Linux and 20 FreeBSD bootable systems on one hard disk.

If you've read my negative comments on dual booting elsewhere on this site, you may wonder why I'd discuss dual and multi booting at all. First, multi booting is only appropriate for special situations. The most common situation for dual or multi booting, is to learn a new OS on a home PC. It is nearly always a poor choice to make the only or primary home PC a dual boot system. Such a system may have much irreplaceable information on it. Whenever you start changing disk partitions and boot records, there is a very real chance of error that leads to irretrievable data loss and even with good backups restoring a damaged disk can be a significant undertaking.

Also existing machines typically have their entire hard disk already allocated. There are programs to manipulate existing partitions with data and create free partitions (FIPS). This is a high risk activity. The link is to an Internet Archive page. From this you can use Google to find various download and documentation sites, but it is not clear that anyone is currently supporting this program. If a primary home PC must be used as a dual or multi boot machine, it will be much safer to add a second hard disk and leave the first relatively untouched. You will need to use a boot loader to boot from the second disk. GRUB will do nicely because it includes a floppy mode, that allows full testing, before committing it to a hard disk.

The risk of data loss is not the primary reason that using a primary home PC as a dual boot machine to learn a new operating system is a poor choice. Using a primary home machine as dual boot machine will never really provide the opportunity to learn the other operating system that is the reason for dual booting. This is because the primary home PC will be too valuable to leave inaccessible for any length of time. As a result, as soon as the initial novelty of the new system wears off, and there is no specific task to perform on it, the home PC will be booted back to the primary system, whatever it may be, and rarely returned to the system which was installed for learning purposes.

FreeBSD, Linux, and OpenBSD, easily fit and run on hardware that has long since been outgrown by Windows. Often a better solution is a keyboard, video, and mouse (KVM) sharing box with an older PC. A good four port KVM costs around a hundred dollars. These avoid the space, cost, and convenience issues of multiple keyboards and monitors. It's really nice to sit at your comfortable computer desk and chair and hot` key to control two or more computers that may be networked together. You'll learn much more this way, because to take full advantage of the learning computer, you'll need to network it with your primary computer and both will always be available.

If the secondary PC has enough disk space, say 5GB or more, you can easily dual boot this machine to learn more than one OS. Smaller disks will work but you need to pay careful attention to partition sizes. If you buy a new low end PC for this, you can put 3 OSs on it. Though you won't be able to network these with each other, each can be networked with the primary machine. If the secondary machine is set up as a server or firewall and not a desktop, you may find that you spend most of your time on it via telnet or ssh from the primary machine.

Where does it make sense to dual or multi boot a PC? Where you want to experiment with more systems or configurations than you have machines for and the disks are set up from the begining as multi boot systems where no data is at risk. This obviously includes the secondary home machine, set up for learning purposes, but could be any test machine. I have 4 "lab" PCs, each with at least one removable hard disk tray and 5 extra hard disks in removable tray inserts. Still I'm frequently looking for a disk to erase for some new install. While writing this I had more trays on order but at about $140 for a disk and tray, it mounts up. Especially since all the newer disks are 30 or 40GB (the smallest now readily available) and two to ten times the size of the original disks, they have huge amounts of wasted disk space. I realized if I could start dual or multi booting these systems, I could probably stop buying new hard disks for some time.

"Partition" has multiple meanings. Its first use is as a hard disk partition that may be referred to as a primary partition, a secondary partition or an extended partition. These disk partitions are sometimes also called MBR or master boot record partitions. An extended partition may sometimes be referred to as and extended DOS partition. FreeBSD refers to all of these partition types as slices. Generally, unless one of the qualifiers, (hard) disk, primary, secondary, extended, or MBR, is used in conjunction with partition I'll use the term slice instead, since it is unambiguous as it pertains to hard disks. A slice is the highest level of organization on a hard disk. A hard disk must have at least one slice, which may occupy the entire hard disk, to be used by an operating system. The Intel PC architecture limits a single hard disk to a maximum of four slices.

The other way that partition is used is to refer to a disk area that has a UNIX device name associated with it and onto which a file system will be mounted. In Linux, hda1 or /dev/hda1 or hda5 or /dev/hda5 will often be referred to as a partition. The OpenBSD counterpart might be wd0a or /dev/wd0a and in FreeBSD is likely to be ad0s1a or /dev/ad0s1a. These are also called partitions. Partition, without qualification, will refer to a UNIX disk area on which a file system will be mounted.

Partition may also be a verb in which case it is the act of dividing a drive into slices or creating operating specific partitions depending on the context.

The PC architecture and each of the operating systems being discussed place constraints on the boot process that limits how the disks can be set up. In the Intel PC architecture, a hard disk can have only four primary, secondary or extended disk partitions or four slices. Traditionally under DOS only one would be primary and the rest secondary or extended partitions. With the open source systems, it's possible that all four will be primary partitions. The difference between primary and extended partitions is important to Linux. FreeBSD and OpenBSD install only into primary partitions. Which partition is active or marked for booting will be important if a boot manager or boot loader doesn't intervene. A hard disk doesn't need to have multiple slices; a single boot, OpenBSD install will only have one slice. A simple FreeBSD system may use only one slice.

The first sector or 512 bytes of a hard disk is often called the Master Boot Record or MBR. It contains, two critical items: the partition table that defines the type and starting and ending location of the four slices that a disk may have and the initial program load (IPL) code. When a PC starts from the hard disk, the BIOS passes control to the IPL. This small piece of code does little more than find the next piece of code to execute, necessary to start the operating system. Damage to the IPL will cause the "DISK BOOT FAILURE" message.

The smallest disk area that can be updated is a sector so updates to either part of the MBR require reading the entire MBR, making the necessary changes at the appropriate location and rewriting the MBR. Any program that doesn't follow the conventions regarding the contents of the MBR can cause serious problems. With the right knowledge and utilities, most damage to the MBR can be corrected, but as a practical matter, a damaged MBR often means an unbootable disk or lost data. The IPL is comparatively easy to fix compared to the partition table because it holds relatively standard information. Unless a separate accurate record of the partition table contents happens to be available, damage to the partition table will likely result in loss of data even though the data still most likely is on the disk. If the partition table contents have been recorded separately, recreating it is the same as defining it the first time, but if not, relatively few have the skills and tools necessary to successfully identify the slices as they exist on the disk.

Setting up the disks for FreeBSD, Linux, or OpenBSD is typically a two step process. Each provides an fdisk program that can do the first step which is to create the slices and define their types and physical location on disk, i.e., which cylinders the slice occupies. All can delete slices belonging to other OSs whether they are used or not.

Each of the slice types is identified by a hex number: a5 = FreeBSD, a6 = OpenBSD, 82 = Linux swap, 83 = Linux files, 05 = extended. FreeBSD's fdisk uses decimal numbers instead; they are: 165 = FreeBSD, 166 = OpenBSD, 130 = Linux swap, 131 = Linux files, 5 = extended. There are many other partition types but these are the only ones needed to install FreeBSD, Linux, or OpenBSD. A5 (or 165) also identifies NetBSD slices.

The second step of setting up a disk is dividing a slice devoted to the operating system into OS specific partitions, setting the size and location of these partitions, and assigning file system mount points to the device names that directly correspond to the partition identifiers seen in the setup program, which is disklabel in FreeBSD and OpenBSD, and Disk Druid in Red Hat Linux.

Linux confuses the issue, by mixing OS specific partition creation with slice creation. The normal (single boot) method for installing Linux is to consume three primary partitions (one for each device) then create an extended partition for any additional devices and file systems that will be used. Assuming a system contains only one IDE hard disk, Linux names the primary or extended partitions hda1 - hda4 or /dev/hda1, etc. A second hard disk might be hdb or hdc (if a CDROM drive is hdb); the first partition would then be /dev/hdc1. Linux system partitions located inside an extended partition start with hda5, even if there are unused primary partitions. Fdisk can create 12 subpartitions in the extended slice for a total of 16 devices per disk hda1 - hda16. If hda4 is an extended partition, it can't be used directly but is just a container for hda5 - hda12.

All the fdisks will allow you to create slices where the number of the slice does not correspond with it's physical location on the disk. While the computer won't care, you'll find it easier to remember which partition is which if the partition numbers increase with the partition location on the disk. If you skip the first part of the disk and install at the end, as some scenarios below suggest, Linux and OpenBSD's fdisks will let you skip the low number partitions and create the higher ones first. If the front of the disk is left empty, skipping the partition numbers that would typically be located there, makes sense.

Red Hat Linux confuses the partitioning process a second way. The fdisk step is optional, and Disk Druid which is used to assign mount points to partitions created by fdisk, can dynamically create the necessary partitions including primary, extended, and Linux specific partitions in the extended partition. With Disk Druid you have no direct control of where the partitions are located. There is an automated Disk Druid option where partition sizes and mount points are determined by Disk Druid based on the available disk space. Disk Druid can delete any slice or partition it displays.

OpenBSD's disklabel allows up to 16 partitions. It sees all primary partitions that exist. If Linux has been installed, OpenBSD's disklabel sees all Linux partitions inside an extended partition but it does not list the extended partition container. It also uses a reserved c: partition, that represents the entire hard disk. I: through p: may already assigned at the start of disklabel when Linux is installed before OpenBSD. With c: also reserved this leaves seven OpenBSD specific partitions; these are a:, b: and d: - h:. Assuming this is the first IDE disk, these will become wd0a - wd0h or /dev/wd0a, etc. SCSI disks will be sd0a - sd0h. A second hard disk would be wd1a. Since OpenBSD's disklabel will only see already used partitions when other systems are already present or at least slices reserved for them, seven OpenBSD specific partitions should be sufficient. One or more foreign partitions can be deleted and the letters used to assign space within the OpenBSD slice.

FreeBSD's disklabel only displays slices reserved for FreeBSD (or possibly NetBSD as they use the same slice type identifier, hex a5 or decimal 165). Within these slices, FreeBSD's disklabel can delete any existing slices and create new ones from any free space. Disklabel allows seven partitions per slice. As a FreeBSD install may use more than one slice, a single FreeBSD system may have more than 7 partitions.

FreeBSD labels hard disks from ad0. Within a hard disk the four slices are s1 - s4. The partitions within a slice are named a - h so the first FreeBSD partition in the first slice of the first hard disk is ad0s1a or /dev/ad0s1a. The next six would be ad0s1b - ad0s1h (skipping ad0s1c as FreeBSD also reserves c). The fourth partition of the third slice on the second hard disk would be ad1s3d or /dev/ad1s3d. Since FreeBSD assigns the letters dynamically, sometimes changes them, and doesn't display physical location, 'a' is logically first and 'b' second but there is no way in disklabel to be sure the physical order matches.

The disk partitioning programs that come with FreeBSD, Linux, and OpenBSD have specific limitations and problems. Most problematic are OpenBSD's fdisk, as it is the most difficult to use and lacks any integrity checking, and disklabel that may not work properly in the presence of previous install data, other systems, or in other than the first slice. Details are provided on the OpenBSD install page and elsewhere when appropriate. Once another system is installed, OpenBSD's fdisk may render another previously bootable system, unbootable when it saves changes.

Linux's fdisk will only write changes that are compatible with the established conventions but it may become inoperative in the presence of BSD disklabel data from previous installs and unable to write any changes to disk.

FreeBSD's fdisk is by far the most robust of the three systems. FreeBSD's fdisk can always use any unallocated hard disk space, can remove any slices created by any other partitioning software, and can create slices for itself or any other system that will be recognized by all the other systems. The FreeBSD install gives you an explicit choice of installing its boot manager, installing a standard MBR without a boot manager, or doing nothing to the MBR, (or more correctly, the IPL part of the MBR).

Normally, FreeBSD won't commit any changes to disk from it's fdisk, MBR (IPL) software, or disklabel until a distribution is selected and an actual install started. Both FreeBSD's fdisk and disklabel programs have hidden write options. Pressing 'w' in either will bring up a warning dialog that's defaulted to not saving any changes. If you continue, your changes are supposed to be immediately written to disk.

Thus, it should be possible to use FreeBSD's easy to use and powerful disk partitioning software as a limited, general purpose partition manager, even when you don't plan to perform an install. The only time I tried to use FreeBSD this way, OpenBSD did not see what FreeBSD should have written. The disk appeared completely empty, as it had been initialized by a partition manager, before I tried to create the slices with FreeBSD's fdisk. In the many installs I've done, experimenting with dual and multi booting, this is the only anomalous behaviour I've seen from the FreeBSD programs.

FreeBSD's install software can install FreeBSD multiple times on the same hard disk. Four independent OS installs in four slices on the same hard disk is easy. FreeBSD can install multiple systems in a slice, share partitions between different systems, and use two or more slices or parts of them for a single system. If multiple systems are created in a slice, it will need GRUB to boot any "/" file system which is to be mounted on a partition that does not end with 'a'.

Perhaps the best known issue related to booting is the 1024 cylinder issue. Traditional BIOSs can not boot a disk where boot loading programs reside beyond the 1024 cylinder limit. Recent BIOS usually do not have this restriction. FreeBSD and Linux no longer have this limit, so on machines with newer BIOSs, they can be booted from any disk location. OpenBSD remains constrained by this limit, and GRUB and other boot managers cannot get around it. With older BIOSs, i.e., those that do not support LBA mode, FreeBSD and Linux may also be limited to booting from the first 1024 cylinders.

A BIOS may support LBA mode, but not understand hard disks over 32GB, that are now the norm in replacement and upgrade disks. In this case, the disk won't be usable, unless a BIOS upgrade is available. Mother boards made in 2000 may, and those made before, will likely have the 32GB limit. If the motherboard isn't too old, there may be a flash BIOS upgrade that can fix the problem. If you have a BIOS that fits this description, go to the web site of the motherboard manufacturer and look for bios updates. Like many things, the upgrade procedures may be described in such a way as to sound complicated. The actual mechanics of upgrading a BIOS are likely to be very simple once you get past the confusing descriptions.

LBA has been around long enough that if your computer does not support it, you probably can't fix the problem. BIOSs this old probably wont support flash upgrades either. Still before giving up, it can't hurt to check the manufacturer's web site to see if any upgrades are an option, and if they address the problems you have, specifically lack of LBA support.

The traditional approach to dual booting systems limited to booting from the first 1024 cylinders, is to put the boot part of both systems inside these cylinders. Depending on the sizes of the disks involved and the sizes of the systems to be installed there are two approaches that can be used. If the total install size of one of the systems is small enough that it can fit entirely within the 1024 cylinders and leave room for the "/" or "/boot" partition of the second, also within the 1024 limit, the install is simple. The smaller system goes first. It may actually be installed first or the space for it may be reserved by creating an empty slice in the fdisk program of the other system.

If the smaller system is too large to allow the boot part of the other system inside of 1024 cylinders, then one of the systems must be split into two pieces. A small slice goes first on the disk and contains the system A partition and with the boot files for that system. Then B occupies a large slice that crosses the 1024 boundary, but the "/" or "/boot" file system is placed first and is small enough that all of it fits inside of the 1024 cylinder limit. Strictly speaking this may not be necessary, but because the exact location of the critical files cannot be accurately predicted, this ensures the necessary boot files will be fully within the 1024 cylinders. Then the rest of A goes after B in a third slice. All that's required is determining how much disk space each cylinder represents and doing the calculations to determine how far inside the 1024 cylinder limit, the boot portion of B needs to be to ensure its all inside of 1024 cylinders. There may be a lot of latitude. On the 30 and 40GB disks I've been using the translated 1024 cylinders include between 7 and 8GB which is enough room to store several FreeBSD, Linux, and OpenBSD installs.

Since FreeBSD and Linux no longer have the 1024 cylinder limit. No combination of these two and OpenBSD requires the two slices for one system approach. Only on an old PC where the BIOS forces the issue will this still be a problem. Since I have no PCs with such BIOSs to test, I can only describe what needs to be done but can't recreate and test the situation. It's nothing more than doing the calculations to determine the slice sizes for one system that has two slices instead of one and another that has a slice that comes between these two and ensuring the start of the one slice system is sufficiently inside the 1024 cylinder limit. Then it's simply a matter of entering that information into the appropriate fdisk. Though most of what follows will still be relevant, I won't address the mechanics of splitting systems to deal with the 1024 cylinder limit further. In all cases, the installs described will be the simpler setups where each OS is kept in contiguous disk areas, whether this is one or multiple slices.

Disk sizes have increased much faster than BIOSs and the Intel PC architecture could keep up with. The size of a hard disk is determined by the number of cylinders (concentric magnetic tracks aligned vertically above and below on multiple disk platters), the number of platter surfaces that contain data and normally correspond one for one with the magnetic heads that read the data, and the number of 512 byte sectors in a track. Old disks had the same number of sectors on the inner short tracks as the longer outer tracks; today's disks store more data on the longer outer tracks. BIOSs can only work with a fixed number of sectors per track so today's IDE and SCSI disks perform translations from the real physical layout of the disk to simpler ones the PC BIOSs can understand. Typically many cylinders and few heads become relatively few cylinders and many heads.

Where the 1024 limit exists, it is based on the translated values seen and used by fdisk. An OpenBSD system in a slice between cylinders 1000 and 2000 on a 30GB drive could boot. This was far beyond the location of the physical cylinder 1024 but /dev/wd0a was limited to 25MB keeping the '/' file system within 1024 translated cylinders. When the slice was moved to be between cylinders 1025 and 2000, the boot process hung as soon as it started after displaying "Using Drive: 0 Partition: 3". I tried to force it with GRUB and got the common "bad magic" message which typically occurs whenever there is a problem in the /boot file.

Besides needing its boot files within the 1024 cylinder limit, OpenBSD has another limitation. It must be installed in a single contiguous slice. As disks gets very large, this tends to anchor OpenBSD towards the lower number cylinders. This doesn't have to be true as either FreeBSD or Linux could be installed before OpenBSD, and on a large disk might be much smaller than OpenBSD. OpenBSD could extend to the end of the disk as long as its boot files were inside the 1024 limit. Both FreeBSD and Linux can be split into two or more slices and they do not need to be adjacent.

Top of Page - Site Map

Copyright © 2000 - 2014 by George Shaffer. This material may be distributed only subject to the terms and conditions set forth in http://GeodSoft.com/terms.htm (or http://GeodSoft.com/cgi-bin/terms.pl). These terms are subject to change. Distribution is subject to the current terms, or at the choice of the distributor, those in an earlier, digitally signed electronic copy of http://GeodSoft.com/terms.htm (or cgi-bin/terms.pl) from the time of the distribution. Distribution of substantively modified versions of GeodSoft content is prohibited without the explicit written permission of George Shaffer. Distribution of the work or derivatives of the work, in whole or in part, for commercial purposes is prohibited unless prior written permission is obtained from George Shaffer. Distribution in accordance with these terms, for unrestricted and uncompensated public access, non profit, or internal company use is allowed.