Dual and Multi Booting FreeBSD, Linux, and OpenBSD

GNU GRUB Boot Loader

• Boot Sequence
• GRUB Disk Nomenclature
• Booting Multiple Disks
• Installing GRUB
• GRUB Boot Floppy
• GRUB Configuration
• GRUB and Linux (RH 7.2 & 7.1)
• GRUB and FreeBSD
• GRUB and OpenBSD

GNU GRUB is a powerful open source, operating system independent, boot loader for Intel architecture PCs. It can be compiled on, installed on, and boot FreeBSD, Linux and OpenBSD, as well as many other operating systems. It can boot systems from any hard disk or floppy disk in the system. If the BIOS and operating system don't prevent it, it can boot systems from any hard disk partition including beyond 1024 cylinders and including any FreeBSD's disklabel partition or Linux's extended partition.

(NOTE: If used with OpenBSD, be sure to use the OpenBSD related commands presented below; the official GRUB documentation is almost three years out of date as regards OpenBSD and the example commands are not even close to what works.)

A particularly useful feature for testing dual and multi boot installs is that it can easily be installed on a bootable floppy which can be used to test the command syntax necessary to load each system. This can be done without committing any changes to disk, specifically not overwriting the hard disk's MBR. Once the necessary commands are determined with use of the floppy, they can be saved in a configuration file on any of the systems to be booted and the MBR updated with a copy of GRUB that knows where to find the configuration file and other pieces it needs.

The first sector of the first hard disk in a system has, in addition to the partition table, executable code that is the first stage of the boot process or the IPL (initial program load) code. The standard code is what's created when the DOS "fdisk /mbr" command is executed. When the BIOS settings are such that the hard disk is set to boot (or the hard disk comes after the floppy but no floppy disk is in the drive), the BIOS passes control to the IPL or first stage code. The standard code loads, determines which is the active partition and passes control to executable code in or near the first sector of the active partition. What happens next will depend on the operating system. With the older DOSs the actual executable core of the OS was located at the very beginning of the disk and loaded directly from the first stage loader. More typically a second stage boot program will load that in turn knows where to find the kernel or core components of the OS and then start them.

When GRUB or another boot loader is installed, the boot loader's code will replace the standard IPL code. It has to fit into the same 512 bytes shared with the partition table. The main difference between the boot loader and the standard IPL code is where control goes next. In the case of GRUB, control is passed to GRUB's stage1_5 loader. When a hard disk or floppy is installed with GRUB as a boot loader, the appropriate stage1_5 file, which is file system specific and between 8 - 11K, is put on the first track of the disk device, starting at sector 2. Stage1_5 is probably modified by the setup process to include the GRUB root device that will be discussed next in the GRUB Install section. Stage1_5 then loads stage2 from GRUB's root device. The stage2 loader is 120K and too large to fit in the first track of a disk. The grub.conf configuration file is also stored on the GRUB root device. When the stage2 runs, it reads the grub.conf file and sets up the menu with the systems defined in the grub.conf file. If no grub.conf file can be located, the stage2 loader goes directly into command prompt mode, which can also be accessed from the GRUB menu by pressing 'c'.

Assuming the grub.conf file has the appropriate commands to start the installed systems, GRUB will wait for an amount of time specified in grub.conf, start the user selected system when the user makes a selection, or if no selection is made, start the default system when the time to wait expires. How GRUB starts a system depends on the system. With some systems like Linux, it directly loads the kernel or loads a standard boot loader as it does for FreeBSD. With most systems it chain loads the system, meaning it does what the standard IPL loader does except that it provides other alternatives before doing this. When the system to be booted must be chain loaded, GRUB typically marks that system's partition active. Where the standard loader can only work with the partition that is already set as active, GRUB allows this to be controlled. It then passes control to a program at a known location on the boot partition, typically at +1 sectors or the second sector. This is OS specific code that completes the boot, in the normal manner for the selected OS.

GRUB identifies hard disks by "hd" and floppy disks as "fd" followed by a number starting from 0. In commands, the disk references are always enclosed in parentheses, thus the first hard disk would be referenced as (hd0). Disk partitions are separated from the hard disk by a comma and identified as 0 - 3 for the primary partitions and 4 and up for partitions inside an extended partition. FreeBSD numbers its slices from 1 to 4, and Linux its partitions from 1 to 16, rather than starting at 0, so you'll need to reduce a FreeBSD slice or Linux partition number by one to get the right disk partition number. If the partition is a BSD disklabel "subpartition" in GRUBs terminology, these are separated by a comma and start from 'a'. Thus (hd0,2,a) would point to FreeBSD's /dev/ad0s3a or the 'a' partition of the third slice on the first hard disk.

GRUB uses these device specifiers for two purposes: first to identify the location of its configuration files and then to identify the system to be booted. The default location for GRUB's configuration file is in /boot/grub/grub.conf on the partition defined as GRUBs root (see below). The default configuration file can be installed on either Linux or FreeBSD systems but a different location is needed on OpenBSD systems because the default location conflicts with the OpenBSD standard "/boot" file. GRUB has syntax for specifying files by name or block lists of physical sectors where the file resides. This is not necessary for our purposes. If you need more information use "$info GRUB" on a system GRUB is already installed on, such as Red Hat Linux 7.2, or see the online documentation at http://www.gnu.org/software/grub/manual/. When this URL was updated (Oct. 2004) the Grub discussed here has been renamed by GNU.org to Grub Legacy, and a new product Grub 2 is under development. There is no documentation yet for the new product. The common home page for both products is http://www.gnu.org/software/grub/

With GRUB, you might say all you need to do is change any (hd0) reference to (hd1) and all the commands that work on the first disk will work on the second. This almost true. With FreeBSD and OpenBSD, it is true as far as GRUB goes. The GRUB commands for these systems have no other references to the hard disk being used. The commands to start Linux however include a command line argument for the kernel which tells the kernel the device it's booting from, naturally using Linux's device syntax and not GRUB's. This needs to be adjusted as well as would any other disk references for other operating systems.

This leads to another issue. When operating systems are installed they typically contain numerous references in their system files, pointing to the locations of various system elements. As long as FreeBSD, Linux and OpenBSD are installed in the location that they are being booted from there should be no significant issues (but I've only installed one OpenBSD system on a second hard disk to test this; on a relocated disk, each started to boot but failed during the boot process). It's an entirely different matter to move a system installed at one location to another. Typically you can move a disk and mount its file systems from compatible operating systems for reading and writing, but not booting, as all the internal references to devices are wrong at the changed location. It may be possible to find and adjust all such references (while booted from another system) but it's probably quicker to install a system from scratch at a new location than to move it.

If you intend to boot from the hard disk, then the executable copy of GRUB still needs to be installed onto the hard disk that the BIOS will pass control to at boot time which is nearly always whatever disk is recognized as the first disk (or the one you currently boot from). Another possibility is to make no changes at all to the first hard disk, and to make a GRUB boot floppy with a menu for booting any systems that may be installed on the second or additional hard disks. This is more practical than it sounds. GRUB and its menu load from a floppy much faster than FreeBSD, Linux, or OpenBSD, boot from floppy.

Eventually, the GRUB floppy should contain an entry and commands to boot any system installed on the first hard disk. Even if you start from a floppy, if your setup is reasonably constant, you'll want to install it to hard disk at some point and before you do this, the commands to boot any system on the first disk should be tested. When you do install GRUB to the first hard disk, you can use any system to store GRUB's files on. If you store them on the second disk and later remove this disk, GRUB won't finish loading because it can't access its stage2 file.

Before installing GRUB, be sure you know how to restore the first disk to a bootable state and document this in case you ever remove the second hard disk (or it has a hardware failure). If you made a GRUB boot floppy that booted systems on the first disk this will still work. You should probably restore the first system to its prior bootable state, before removing the second disk, if that is where you place the GRUB directory. For a single boot DOS or Windows system, restoring a bootable state may require no more than "fdisk /mbr". For a multi boot system, you'll probably reinstall any previous boot loader. You could stay with GRUB, but if you were going to do this, it would make better sense to install GRUB's stage2 and configuration file on the first disk initially. Installing GRUB in any system except Linux or FreeBSD, may require learning GRUB's syntax to identify specific files or directories.

If you followed my Linux install suggestions, GRUB is loaded in the first sector of the Linux boot partition, /dev/hda5. This insures that Linux is bootable with a Linux boot floppy but only with a floppy and not directly from the hard disk. If Linux is the first installed system, then it makes sense to proceed immediately to set up Linux to boot from the hard disk by installing GRUB into the hard disk MBR.

If Linux is installed after FreeBSD and or OpenBSD and they are bootable, creating a GRUB boot floppy to confirm the configuration and that all systems are bootable from GRUB, before updating the MBR seems prudent. The only downside is that Linux will have to boot from a floppy until GRUB is installed in the MBR.

The instructions for installing GRUB to the MBR will be provided first. Creating and testing a floppy will be covered second.

Installing GRUB into the MBR is simple. Get on the system that has GRUB installed, presumably Linux. If other systems are already installed or you know what disk locations they will occupy (using GRUB's notation), it's probably useful to cd to the GRUB directory, /boot/grub, and update the configuration at the same time you install GRUB.

As root, start the command line version of GRUB with "#grub". You'll see a "grub>" prompt. Enter the find command looking for a key GRUB setup file, "stage1", in the default locations.

grub> find /boot/grub/stage1
    (hd0,4)

The response, (hd0,4), simply confirms what we expected. The GRUB configuration files are located on the first Linux extended partition. Since we installed all Linux partitions into an extended partition, the first extended partition number (Linux's boot partition /dev/hda5) is 4. To install GRUB we first tell it where to find its second stage loader and configuration files. To do this the root command is used.

grub> root (hd0,4)

Then GRUB is placed in the MBR.

grub> setup (hd0)

The MBR goes with the disk not a partition so the first disk is specified. After this, Linux should be bootable from the hard disk. The Red Hat install creates a GRUB configuration file with the correct commands for booting Linux. Several messages should be printed by the setup command. The end of the next to the last should say "... succeeded" and the last line "Done." Type "quit" to exit the GRUB prompt. If the install succeeded, GRUB should now function as a boot loader running from the hard disk. It will present a boot time menu as defined in the GRUB configuration file (located in the GRUB root defined in the just completed hard disk install).

To make a boot floppy, the online GRUB documentation presents two different methods, in the manual section "Creating a GRUB boot floppy", and in the GRUB FAQ which is the simpler method. These instructions are slightly modified from the FAQ instructions. Insert a new floppy or one which can be erased into the drive and create a new file system on the floppy with "mke2fs /dev/fd0". Mount the floppy with the new file system, "mount /dev/fd0 /mnt/floppy". If you've built and installed GRUB on FreeBSD or OpenBSD, modify these OS commands appropriately. Create a new /boot/grub directory on the floppy and copy the files stage1, stage2, and menu.lst from /boot/grub to /mnt/floppy/boot/grub. If you want the boot floppy to go directly into the system boot selection menu, then also copy grub.conf By pressing 'c' in the selection menu you drop to the GRUB command prompt. As you may not remember the commands to load a specific system this may be convenient since it does not hinder access to the GRUB command prompt. Unmount the floppy but leave it in the drive.

Then run GRUB interactively and use the following commands

grub> root (fd0)
grub> setup (fd0)
grub> quit

These commands correspond directly with those used to set up the hard disk MBR. First the root command tells GRUB that its files are on the floppy (in the default location), then setup installs the boot programs to the floppy. Assuming setup ended with "... succeeded\nDone." you should now be able to boot from this floppy. If you included grub.conf, a menu of the systems listed for booting in grub.conf should be displayed. If so, you can get to the command prompt by pressing 'c'. If there is no grub.conf, then GRUB will go directly to the command prompt. In this case, you'll need to know the exact filenames and locations of the two Linux boot files to boot Linux. You can test booting FreeBSD and OpenBSD with the simpler command lines for those systems if they are installed.

When you've installed Linux with GRUB and know where you plan to install any other systems, you may want to setup grub.conf once, for all the systems that will be booted via GRUB. The worst that can happen is that you'll get a command wrong and the system won't boot. This is where the floppy will come in handy. You can boot using it and try the menu. If one or more selections don't work, go into command mode, and try the same commands. If you can't remember the commands, GRUB includes a "cat" command. "grub> cat /boot/grub/grub.conf" will display the configuration file from the floppy; long files will scroll off the screen. You can observe GRUB's error messages and experiment until you get valid commands; for any other systems including other Linux variants you may have to do some research on the system's boot process. When you successfully boot from the floppy, you'll know what corrections need to be made to the grub.conf file. If you type "help" in GRUB, you'll see two full columns of available commands. Obviously many of GRUB's many features are not being covered here, only the essential mechanics to boot FreeBSD, Linux, and OpenBSD with it.

Below is a grub.conf file, less the leading comments, to boot a disk with one OpenBSD, two FreeBSD and one Linux system.

default=0
timeout=10
#splashimage=(hd0,4)/boot/grub/splash.xpm.gz
title Red Hat Linux (2.4.7-10)
        root (hd0,4)
        kernel /boot/vmlinuz-2.4.7-10 ro root=/dev/hda5
        initrd /boot/initrd-2.4.7-10.img
title OpenBSD 3.0
        root (hd0,0)
        makeactive
        chainloader +1
title FreeBSD 4.4 #1
        root (hd0,2,a)
        kernel /boot/loader
title FreeBSD 4.4 #2
        root (hd0,3,a)
        kernel /boot/loader

The bulk of the file are the four sections that start with the word "title". Everything on the same line following "title" is simple text that will be displayed in GRUB's boot menu to identify the system to be booted. The following lines, until the next title lines are the commands necessary to load the system. The order the title sections appear is the order the prompts appear on the menu.

Before the system specific commands are general configuration commands. The "default" command tells GRUB which system to highlight and boot if a user doesn't make a manual selection. The numbers, starting with 0, refer to each "title" section. With this configuration file, to make OpenBSD the default in this grub.conf file, change the 0 to 1. You could move another title section first, to display it first and make it the default, leaving default=0.

You can also have GRUB save the last loaded system as the default for the next boot. Change the "default=0" line to "default=saved" and add the "savedefault" command (no arguments) as the last command in each title section. When you switch to the savedefault mode, the first title section will be the default on the first boot. GRUB saves the current selection just before initiating the boot sequence. By using the savedefault command in two or more title sections and leaving it out of one or more title sections you could create a menu that would allow the default to switch between some systems but never be set for others.

The timeout is how many seconds GRUB waits before loading the default system. If any keys are pressed in this time, GRUB will wait until an explicit selection is made.

The commented splashimage line would normally cause the Red Hat boot page graphic to be displayed. The file specification syntax is simply the absolute path the to the file, starting from the root and with the path following the root without any intervening spaces. The image does not need to come from GRUB's root location. I don't object to the Red Hat splash screen but that when you expect to use the command mode, the commands are much harder to read against the Red Hat graphic than the default text display, which is light gray letters on a black background.

For the system specific commands, when they are entered interactively at a GRUB prompt, they must be followed by a separate explicit "boot" command to start the system just specified. When they are entered into a grub.conf file the boot is implicit. As soon as the explicit commands are executed, boot is executed. Don't forget to type boot in interactive mode or Grub will sit idle until you do something.

title Red Hat Linux (2.4.7-10)
        root (hd0,4)
        kernel /boot/vmlinuz-2.4.7-10 ro root=/dev/hda5
        initrd /boot/initrd-2.4.7-10.img

# The following works to boot Red Hat 7.1 installed on
# /dev/hda7
title Red Hat Linux 7.1
        root (hd0,6)
        kernel /boot/vmlinuz-2.4.2-2 ro root=/dev/hda7

The Red Hat 7.2 Linux commands are exactly as they have been set up by the Red Hat install. Since they work, I've seen no reason to look for alternatives. Regardless of how you set up Red Hat 7.2 Linux, the installed GRUB commands should be correct.

If you are using a distirbution other than Red Hat Linux, you may need to adjust the root to reflect the actual Linux boot partition. You'll also need to adjust the exact filename of the kernel and its location. As for the rest, including the initrd (ram disk loader), you'll need to read the GRUB and possibly some distribution specific documentation to make any necessary adjustments. Since the initrd line is not necessary for Red Hat 7.1, it may not be necessary for other Linux distributions.

The one Red Hat 7.1 install I did, overwrote GRUB in the MBR with LILO, and GRUB had to be restored as described elsewhere in this section.

title FreeBSD 4.4 #1
        root (hd0,2,a)
        kernel /boot/loader
title FreeBSD 4.4 #2
        root (hd0,3,a)
        kernel /boot/loader

GRUB's FreeBSD loading commands are simple and have little variation. All that will change will be the root location which can be any disk in the system and any slice from 0 - 3. The disklabel subpartition letter will also vary for advanced FreeBSD installs, that include multiple bootable systems in a single slice. The second command is supposed to be the same for all recent versions of FreeBSD. In the examples above, which came from the first quad boot system I created, the Linux extended partition was in the second partition and the two FreeBSD partitions came last so the 2 and 3 partition indicators were correct for this system.

title OpenBSD 3.0
        root (hd0,0)
        makeactive
        chainloader +1

OpenBSD was installed into to the first disk, partition 0, so the root is what's expected. The GRUB documentation describes OpenBSD as one of the officially supported systems and says the boot commands are identical to those provided for NetBSD. This is simply wrong or more accurately badly out of date. Apparently this was true up through OpenBSD 2.5; at least that's what's indicated in the GRUB e-mail list archives. The NetBSD commands did not work for me and I tried many variations. GRUB archive searches provided the answers. As OpenBSD diverged from NetBSD, they apparently changed something important in the boot process in 2.6. There were messages from 2.7 and 2.9 users giving exactly the same errors I was seeing in 3.0. Commands that boot OpenBSD were provided in the answers.

GRUB aspires to be a universal boot loader. As such it provides the first two stages of the boot process then directly loads the kernel or other OS specific boot files. A number of systems use boot approaches that are not compatible. To deal with these systems, GRUB also functions as a simple boot manager and not a full loader. To boot these systems GRUB marks their partition as active and passes control to the normal OS specific, boot loader comparable to GRUB's stage1_5 loader. This is called chain loading. The GRUB developers clearly believe this is an inferior method. Exactly the same sequence of commands and syntax as are used for OpenBSD, should work for Windows systems and OS/2; all that should change is the root location.

From a user perspective, the only difference is the completely different command syntax. Because chain loading is being used with OpenBSD, only the hard disk and primary partition are provided, and not the disklabel subpartition that would be expected of a BSD system. Depending on disk size, the primary partition is likely to be 0 or 1, but on smaller disks that don't go much beyond 1024 cylinders, may be anything from 0 to 3. The other two commands should be identical for all OpenBSD systems from at least 2.7 and later.

Two or more OpenBSD systems can be installed and booted from a single hard disk with GRUB's assistance. Grub.conf is updated to contain two (or more) parttype commands as shown below. The first two lines in each title section, switch the slice types so neither OpenBSD system ever sees the other; they are necessary. If two a6 slices exist, only the first will ever boot regardless of the following GRUB commands. At boot time, the user sees two prompts, and booting two copies of OpenBSD becomes just a matter of the using the right GRUB commands. With GRUB, it's no more difficult than booting any other two systems.

title OpenBSD 3.0 One
   parttype (hd0,1) 0xa8
   parttype (hd0,0) 0xa6
   root (hd0,0)
   makeactive
   chainloader +1
title OpenBSD 3.0 Two
   parttype (hd0,0) 0xa8
   parttype (hd0,1) 0xa6
   root (hd0,1)
   makeactive
   chainloader +1

Top of Page - Site Map

Copyright © 2000 - 2014 by George Shaffer. This material may be distributed only subject to the terms and conditions set forth in http://GeodSoft.com/terms.htm (or http://GeodSoft.com/cgi-bin/terms.pl). These terms are subject to change. Distribution is subject to the current terms, or at the choice of the distributor, those in an earlier, digitally signed electronic copy of http://GeodSoft.com/terms.htm (or cgi-bin/terms.pl) from the time of the distribution. Distribution of substantively modified versions of GeodSoft content is prohibited without the explicit written permission of George Shaffer. Distribution of the work or derivatives of the work, in whole or in part, for commercial purposes is prohibited unless prior written permission is obtained from George Shaffer. Distribution in accordance with these terms, for unrestricted and uncompensated public access, non profit, or internal company use is allowed.