GeodSoft logo   GeodSoft

Hardening OpenBSD Internet Servers
Building a Custom Kernel

The kernel contains many of the operating system features. Like external utilities, not all are needed or appropriate on all machines. The GENERIC kernel tries to be everything to everyone. Building a custom kernel allows removal of unneeded features, and those which may be useful to an intruder after they've already gained some degree of access. Detailed step by step instructions are included and the pros and cons of various choices discussed.

The preceding contents page used to contain the following statement regarding the GENERIC kernel: "It also includes support for filesystems, debugging and other capabilities that may not be necessary or even appropriate on a production server." Marc Espie wrote to me on behalf of the OpenBSD development team regarding this statement and said:

This is not quite true. GENERIC is appropriate for a production server. In fact, the OpenBSD project encourages the use of GENERIC above all other combinations. GENERIC is the most tested kernel. Some bugs are hard to reproduce with a different kernel, and we routinely look at GENERIC bugs *first*, before trying to reproduce other combinations.

By building custom kernels, you run the risk of getting entangled into a bug that no-one else has seen before---including security bugs.

I can't argue with what Marc said because it's potentially true not only of building custom kernels but of all the hardening techniques discussed in these pages. Every one in some way creates a non standard system and any program that depends on a standard setup that has been altered, will fail in some manner when it attempts to make use of a changed system. A very simple example is read-only filesystems; they could break a rootkit script but are more likely to break a normal application. The Analog web analysis program failed when I made /usr a read-only filesystem on my OpenBSD web server.

Of the various techniques discussed, building custom kernels is one of the more time consuming steps, is relatively difficult to test well and therefore comparatively high risk, and certainly provides smaller gains for the effort expended than most of the other techniques discussed. A specific example is that for some months now my production firewall has been running on a kernel that was built with the GPL_MATH_EMULATE option disabled. The 3.0 GENERIC kernel now contains a comment that this is a required option. I've seen no problems on my firewall, but if various applications were run on it, I suspect that some of them would produce unexpected results.

Still the basic principle behind hardening system is removing unneeded functionality and turning general purpose systems into limited or single function machines. The principle is just as applicable to selecting which kernel options are used as what network services are run. If security is the highest priority, custom kernels should be considered, but if If cost effective security is the top priority you'll get more results for the effort by disabling unneeded services, using strong passwords, avoiding insecure protocols like telnet and building a carefully crafted, custom firewall.

A custom kernel can be built at any point prior to removing the development tools; it can also be done after file removal if the files are removed and the CD ROM with the executables is created as described in these pages. I've found however that the more changes are made to a system, the more likely a kernel build is to fail. In particular the build process seems very sensitive to any changes to the default root environment. If a custom kernel is going to be used, I recommend that this be the very first change made following a fresh install. I was somewhat intimidated by the idea of building custom kernels initially. Under 2.7 it was actually very easy. 3.0 was also very easy but it was the very first thing I did following a basic install. It also seems to go more smoothly if it is done on a local console rather than via a network connection.

Under 2.8 I encountered problems building the kernel that took some time to solve. I had changed the default root login files. I had to restore these to their original settings to get the build process to complete. I never isolated what change caused the problem. In addition, if I wanted to make the custom kernel remotely via a telnet session, I had to su with the "-l" option or make would exit with an error message before the build was complete.

The first 2.9 custom kernel build seemed to worked without any problems. I never even tried building a GENERIC kernel. I simply merged the options I'd changed previously into the slightly updated kernel configuration and built a new kernel. It built cleanly on the first attempt. Subsequently I found problems, described below in Large Unwanted 2.9 Change that required numerous attempts to get a satisfactory kernel. By then I'd modified the default root environment which I had to restore before a kernel make would complete.

Getting Started

To build a new kernel you need the source code extracted to a suitable location. As root, mount the CD-ROM with "mount /dev/cd0a /mnt". If your CD device is different or you want to mount it at a different point, such as /mnt/cd, change the command accordingly. For 3.0, mount the third CD then cd to /usr/src and use "tar -zxvf /mnt/src.tar.gz  ./sys" to extract the kernel source files.

If disk space is an issue, after the extract, you can cd to /usr/src/sys/arch and remove the architecture specific files that don't relate to your hardware. This will get back about 30MB. Don't remove the other architecture files, if you plan to use CVS to keep your system up-to-date with either the "current" OpenBSD development or latest stable patch branch.

/sys is a link to /usr/src/sys and can be substituted where /usr/src/sys appears. Cd to /usr/src/sys/arch/i386/conf. Substitute your machine's architecture for "i386". Enter the following commands:

# config GENERIC
# cd ../compile/GENERIC
# make depend
# make

If your kernel build completes successfully, it will end with two lines similar to the following except the numbers will vary:

text    data    bss     dec     hex
2871296 421888  817932  4111116 3ebb0c

There should also be a bsd file in the current directory about the same size as the one in /. Having established that you can build a GENERIC kernel, you can return to the ../../conf directory and edit the GENERIC configuration file or copy it to a new file name. I strongly recommend working on a copy so you can use diff to identify all changes you've made. For simplicity I'm going to use NEW from now on.

Changing Kernel Options

After copying GENERIC to NEW edit NEW. Near the top, change the include line that refers to GENERIC to refer to "../../../conf/NEW". Cd to ../../../conf (or /usr/src/sys/conf) and copy GENERIC to NEW in that directory. Kernel options not dependent on architecture are specified in this configuration file.

After making your first set of changes to both configuration files return to the arch/i386/conf directory and repeat the command sequence above substituting NEW for GENERIC. The config step may reveal dependency issues that your edits have created. In other words you may have removed a device or option that a remaining device or option depends on such as removing a USB bus but leaving a USB device. You must fix any dependency problems revealed by config before you can continue.

Execute "make clean" before "make depend" when config tells you it's necessary. When the new kernel build completes, move /bsd to /bsdo and copy your new kernel from compile/NEW to /. Once the original /bsd is moved to /bsdo, it's probably wise to not replace it with any intermediate kernels you build. In other words preserve the distribution GENERIC kernel. Even if you preserve some intermediate kernels keep the original. If at anytime you encounter a repeatable problem, and restoring the original GENERIC kernel eliminates the problem, then you know the problem relates to your custom kernel even if you have no idea how.

Debugging Problems

Before rebooting with a non standard kernel save /var/run/dmesg.boot to a new location. Use this for a list of devices the kernel detected when it booted with a GENERIC kernel. Reboot and test your changes. If you find a problem with the new kernel, restore the setting or settings that are the likely cause and repeat the cycle until you are done.

If a new kernel won't even boot, you can boot from a floppy, and go to single user mode with -s at the boot> prompt or wait until you get the install/upgrade/shell option and pick shell. You can then "mount /dev/wd0a /mnt". Substitute the appropriate device if / is not on wd0a. You can then move or copy /bsdo to /bsd and reboot into the default kernel.

After booting to a custom kernel check that the core system components are working. Check that all filesystems are mounted. Put media in removable devices and check that they still work. Verify that networking is still functioning. Check that multiple local consoles are available. There is no way to be sure everything is functioning but at least verify the major system components are still available after each set of kernel changes or it may be difficult to figure out what change caused a specific problem.

Architecture Independent Changes

Our purpose in building a custom kernel will be the same as most of the other hardening steps: to remove unneeded functionality. As a by product the kernel will be smaller and a little faster than the generic kernel. Aggressive removal of unneeded devices and functions will result in kernels approximately one third the size of the original on i386 systems. The generic kernel includes support for many features that aren't present or used on any particular machine.

I'll begin with the kernel options in /usr/src/sys/conf/NEW that apply to all architectures. Options that are turned off are simply commented out; I like to use a distinctive comment such as #gds# to quickly distinguish options that I've turned off from those that were off by default.

When I started building custom kernels, I missed (or was overwhelmed by) the options(4) man page that describes all the options in some detail. I was relying on the internal comments of the kernel configuration files, some common sense and trial and error. The Kernel Configuration Options section now lists many kernel options with one line descriptions. Some are linked to full man pages. The options(4) man page has typically paragraph length descriptions of most of the kernel options, often with man links to more detailed information. Unfortunately there does not seem to be any single comprehensive and up-to-date listing of all the options.

A reader suggested that the DDB (kernel debugger), KTRACE (system call tracing) and KMEMSTATS (memory allocation statistics) options were not really appropriate for a production machine. I don't know how to use the debugger and the other information is of no use to me. Nothing in the documentation suggests any normal software depends on these. It also looked like these might be of value to a skilled cracker. I've disabled these options in kernel builds since the Spring of 2001 with no apparent ill effects.

I would very much like to turn off LKM, loadable kernel modules as these "allow the system administrator to dynamically add and remove functionality from a running system." This sounds potentially insecure but there is nothing that makes it clear that no ordinary software depends on this. Thus LKM was left on, the default GENERIC setting. A reader informed me after I posted the 3.0 updates that he had three production OpenBSD servers and had been disabling this option since 2.7 with no ill effects. If I build any more custom kernels, I'll probably disable loadable kernel modules.

From 2.7 through 2.9 I'd disabled soft updates (FFS_SOFTUPDATES) because I didn't think the load on my systems warranted using them and I've always associated higher disk performance with lower reliability. A reader informed me that soft updates not only improve performance but actually increase file system reliability (integrity) compared to the standard ffs filesystem. Following the links from the "Soft Updates FAQ" confirmed this and I built an new 3.0 kernel with FFS_SOFTUPDATES enabled.

I enabled soft updates by adding ",softdep" after each "rw" in /etc/fstab. Then I started 5 recursive copies (from each local console) of /usr to different subdirectories of /tmp once then /var twice. After the copies ran for a while, I cut the power. On rebooting, though fsck generated more errors than I'd ever seen before (expected), it had no difficulty leaving consistent file and directory structures in the /var subdirectories. (Those in /tmp were erased latter in the boot as is normal.) Soft updates will be a standard part of all future BSD installs, not just for performance, but for increased disk reliability, i.e., improved security. If fsck can fix the damage left by repeated power losses during the most disk update intensive activity I know how to create, it seems unlikely that I'll ever see an unrecoverable filesystem in normal activity with soft updates on.

Several file system options were disabled. My BSD systems have no Linux partitions so EXT2FS was disabled. The MFS, memory file system looks like it could be of real use on a machine with ample memory and performance issues. It's obviously not being used on my system nor needed so is turned off. The FFS and CD9660 file systems are clearly needed and thus kept. Elsewhere all functions related to NFS will be turned off. Both NFS client and server support have been disabled since the first custom kernel I built.

Both FDESC and KERNFS sound like options that I want to disable but both are "normally executed by mount(8) at boot time." and the documentation simply isn't clear that these are not used by some system process. I've never seen the /kern mount point so assume that this is not being used. I disabled it with no apparent ill effects. For now, I've left FDESC enabled, as it is by default in the GENERIC kernel. Some of the other file system options clearly look like I neither need nor want their facilities. These include NULLFS, PORTAL, PROCFS and UMAPFS. All are removed from the custom kernel.

I've read about UNION for mounting a CD with source code for kernel development without actually copying it to hard disk. Given that the documentation states this is "still experimental and is known to be somewhat unstable." and all the complexities of what you can and can't do with which layers of the UNION mounted file system, it hardly seems worth the bother. If disk space were an issue it might be different but on contemporary computers disk space is not often an issue. The UNION option is disabled.

The pros and cons of keeping MS-DOS filesystem support should be considered. The reasoning is that by disabling MS-DOS filesystem support, you take away the ability to mount a DOS floppy which could be used to load (or remove) software without authorization. The less secure the physical environment is, the more likely I'd be to disable this. Particularly in a small business that did not have a secured computer room, I'd turn this off by commenting out the "options MSDOSFS" line.

In 2.8 and 2.9, I disabled the IPv6 and IPsec options along with the related PULL_DOWN and CRYPTO options and enc pseudo device. I had no expectation of needing these prior the next release of OpenBSD.

In OpenBSD 3.0, disabling IPv6 will cause sendmail to fail when it's started with the default configuration. To allow sendmail to run without IPv6 two lines must be removed from the active file in /etc/mail. They are the two lines that start with "O DaemonPortOptions=Family=inet6,". The quick way is to comment them out with a # sign at the begining of the line. The "proper" way is to edit in /usr/share/sendmail/cf and delete the two lines that start with "DAEMON_OPTIONS(`Family=inet6,"; I don't know why but commenting out will not work here. After these lines are deleted run "#make" then copy the resulting file to /etc/mail/ If you also need to make the DNS related configuration changes discussed with sendmail on the services page, use the proper method and make both edits to the .cm file or the .cm file edits will overwrite any made directly in the .cf file.

Their are two alternate sendmail configuration files. /etc/mail/ does not have any inet6 line but the file does, so presumably it will need to be modified if used as the active configuration file. Both and will accept connections from remote hosts.

I'm anticipating a VPN project in the near future. This will require options described as disabled above. These include CRYPTO, IPSEC, and the pseudo devices, enc and gre. The vlan psuedo-device looks superficially like it might be related but the "man 4 vlan" documentation says this is for use with "compliant Ethernet devices" and thus perhaps an alternate way to set up virtual LANs rather than part of software based VPN. For 3.0 I left these options enabled so that I can use the same kernel on any machine without worrying if it is suitable for a specific project.

My computers will never be connected to anything via PPP or any telephone or serial connections. I disabled all the PPP and tty options and related pseudo devices. SLIP (Serial Line Internet Protocol) was a precursor to PPP and CSLIP or Compressed SLIP was a variation on SLIP so it's very unlikely that anyone will need the sl device. The only new option that I saw in 2.9 is the addition of a "gif" pseudo-device, apparently used for tunneling IPV4 and 6 through each other. As I'm not using IPV6, this was also disabled.

Prior to OpenBSD 3.0 the firewall was IPFiter and the following was applicable. If I were not using OpenBSD machines as firewalls, I'd think about turning off the related options: IPFILTER, IPFILTER_LOG, bpfilter and bridge. Since I have two firewalls and want the ability to quickly turn a web server into a firewall or vice versa, I kept these options. Also, I need the bridge capability for my firewall.

In OpenBSD 3.0 Packet Filter replaced IPFilter so the two lines that included "IPFILTER" have been replaced with the following two lines:

pseudo-device  pf     1  # packet filter
pseudo-device  pflogd 1  # pf log if

Another new option appeared in the 3.0 GENERIC, platform independent file. This is:

option    ALTQ    # ALTQ base

This is for "kernel interfaces for manipulating output queues on network interfaces" and appears related to the altqd daemon, altq.conf file and altqstat utility. Nothing describes in simple English what these actually do but they appear related to "quality of service" functions, i.e., prioritizing network traffic and assuring sufficient bandwidth for specific classes of traffic. I disabled this in my 3.0 custom kernel with no visible ill effects.

In 2.9 the hardware independent configuration file were COMPAT_11 and COMPAT_43 options that are turned on by default in the GENERIC kernel. These apparently alter the way some system calls are handled. In 2.9 the documentation said the COMPAT_43 "option is discouraged". Since they are on by default in the GENERIC kernel, I'm afraid that turning them off could introduce some minor change that would not show itself for some time, at which point there would be no way to trace the altered behaviour back to the kernel change. Thus these were left turned on.

In 3.0 there are some COMPAT_ options that have changed slightly from 2.9. Older, commented out NetBSD compatibility options were removed completely. The OpenBSD COMPAT_23 had been commented out and an new COMPAT_25 added. The FreeBSD kernel compatibility options were unchanged. Like before, these options have been left at their defaults.

Finally (or actually near the top of the file) four options that were active in 2.9, including one that was listed as required, were removed from 3.0 GENERIC kernel. These were TIMEZONE=0, DST=0, SWAPPAGER and DEVPAGER. The lines were removed from the configuration file.

anyarch_conf.txt is a complete copy of /usr/src/sys/conf/NEW as it was updated for 3.0 and anyrmvd.txt has just the lines that were actually removed. Where I don't understand the purpose of an option, I leave it at it's default.

i386 Specific Options

Some of the architecture specific capabilities removed from the kernel were the older CPU types and the binary compatibility options with other related operating systems. During file removal, many programs are removed from the system. We want to make it as hard as possible for an intruder to replace the removed programs. If all the compatibility options are disabled, then the intruder will have to get OpenBSD binaries. Otherwise they may already have or be able to find a Linux or FreeBSD binary that does the same job. This may not seem significant but doing so forces anyone not already familiar with OpenBSD to become familiar with it. It's just an obstacle.

The i386 specific changes between 2.9 and 3.0 were minimal. An option I'd previously disabled, GPL_MATH_EMULATE, is now commented as required and I've made it active again. The UVM option was dropped. A small number of new devices were added. None appeared relevant to my situation, so I made the two changes just mentioned and used the 2.9 i386 kernel configuration file with the changes I'd made previously, described below.

The X Window system is not being installed or used so the XSERVER and APETURE options are disabled.

Also removed were device drivers for devices not being used or not on the system. You should save a /var/run/dmesg.boot created by a GENERIC kernel. This will list devices detected by the kernel and generally devices not on a system don't need to be included in the kernel. The devices are not all real physical devices and there are dependency relationships that you need to be careful about. A device may be present but if you are sure you won't need or use it and nothing on the system uses it, it may be removed from a custom kernel.

Removed devices included all the network cards and SCSI devices not in the system. You need to keep the driver for your network card which will most likely be different than mine. If your machine has SCSI devices you'll need their drivers. If you have multiple machines and more than one type network or SCSI card, I'd leave the all the drivers enabled for all the hardware in use locally, in all kernels. Otherwise you run the risk of trying to diagnose a hardware problem and having to determine if the kernel on the machine supports the hardware being used. For example networking unexpectedly stops on a machine that has not been changed for some time. One might suspect the NIC. A brand new NIC, but a different brand, is installed and it also doesn't work. This would likely lead in a new diagnostic direction but if the kernel supported the first card and it failed and did not support the second card, the failure of the new NIC would be expected. A hardened Internet server should not have or be using a sound system or other consumer oriented devices including mice, joysticks, midi devices, etc.

A pair of SCSI related lines is required, even on systems that have no SCSI devices. The following lines:

cd* at scsibus? target ? lun ? # SCSI CD-ROM drives
scsibus* at atapiscsi?

are required for ATAPI IDE CD ROM drives to work.

Selecting lines to remove was mostly quite straight forward but it was an iterative process, removing groups of related lines, recompiling, then booting with the new kernel to see what worked or didn't. The above two lines were removed and then restored when the CD ROM wouldn't work without them.

In 2.8 a change was made to the network setup that was not obvious. This was the introduction of Media Independent Interface (mii) drivers that work with network cards. 2.9 and 3.0 use the same approach. The following line was required for my network card to work.

ukphy*   at mii? phy ?       # "unknown" PHYs 

One reader suggested that the "tulip" NIC was a "Digital Clone" and that dcphy* was more appropriate than ukphy*. Since the "dc" of dcphy* matched the dc0 network interface this made sense and was tried. This generated " . . . dc0 phy1 not configured" and "dc0: MII without PHY!" boot errors. Networking was restored by returning to the ukpyh* media interface driver. It should be obvious however from the list of more specific PHYs that precede ukphy*, some other NICs will require, or perform better with more specific PHYs.

The machines that this custom kernel is being built for have USB buses but there are no USB devices in the environment. All USB related options were disabled. Cardbus and PCMCIA support was disabled. 2.9 includes several new drivers, mostly USB and SCSI related. As I don't have or expect to use any of these devices, all were disabled. 3.0 added a few more new devices. None were relevant to my environment. i386 specific changes were so few in 3.0 that I used the same config file as in 2.9 with only two general changes mentioned above.

I considered removing the standard serial and parallel interfaces but decided to keep them. I may wish to connect a printer at some point and though there is some chance that my computers could be stolen, there is no chance that an illicit serial or parallel connection could be used to transfer unauthorized software. In a less secure environment this might be worth considering. i386conf.txt has a complete i386 specific configuration file as it was used with 3.0 and i386rmvd.txt has all the lines that were removed.

Previously, the kernel configuration files I used as examples had only one NIC defined. I'd standardized on one model which I used in all my machines. It's no longer available and I've recently bought two different network cards for testing and replacement. This latest configuration file has three NICs and two PHY's defined so that I should be free to use the custom built kernel in any machine or switch NICs and only need to worry about the proper hostname.if file.

Large Unwanted Change in 2.9

The i386 version of 2.9 (and 3.0) includes the following related options and devices that were not in previous versions.

option WSDISPLAY_COMPAT_USL     # VT handling
option WSDISPLAY_COMPAT_RAWKBD  # can get raw scancodes
option WSDISPLAY_COMPAT_PCVT    # emulate some ioctls

pckbc0     at isa?    # PC keyboard controller
pckbd*     at pckbc?  # PC keyboard
pms*       at pckbc?  # PS/2 mouse for wsmouse
pmsi*      at pckbc?  # PS/2 "Intelli"mouse for wsmouse
vga0       at isa?
vga*       at pci? dev ? function ?
pcdisplay0 at isa?    # CGA, MDA, EGA, HGA
wsdisplay* at vga? console ?
wsdisplay* at pcdisplay? console ?
wskbd*     at pckbd? console ?
wsmouse*   at pms? mux 0
wsmouse*   at pmsi? mux 0

# The next two lines were about 20 lines further down
wsmouse*   at lms? mux 0
wsmouse*   at mms? mux 0

# These next two were the next to the last lines
# and about 200 lines below the others.
# mouse & keyboard multiplexor pseudo-devices
pseudo-device  wsmux    2

First I tried leaving only those I thought I wanted but got config dependency error messages. I restored everything but the mouse options that I did not expect to use. I found that I was getting unexplained keyboard lockups and sometimes a character would start repeating until the keyboard buffer overflowed. The kernel itself was still running because I could reboot the machine from a network session. Rebooting was the only way to get the keyboard back. Once I had to power down and then verified the cables were all securely connected. I could even remotely kill processes on the displayed console until a login prompt returned but still the keyboard would not respond.

I returned to the GENERIC kernel and it did not seem to exhibit the keyboard problems. I then systematically tried different combinations of the above options and devices, starting with what I thought would be the minumum that I wanted (multiple local consoles). I eventually tried over twenty combinations, repeating some of the earlier ones I'd not documented. Everyone resulted in config dependency errors, make failing with undefined wsdisplay related functions, only a single local console, keyboard lockups or uncontrollable repeating characters. It wasn't until I restored everyone of the above GENERIC options, including the mice related ones I could see no need for, that I got a kernel that provided multiple local consoles and did not lock up. A couple of times I had all the closely grouped options and still had problems. It wasn't until I searched the entire file line by line and found and re-enabled the wsmouse* at lms? and mms? separated by about 20 lines from the others and the wsmux psuedo-device separated by about 200 lines from the others that I finally got what appears to be a stable kernel. If it wasn't for the "ws" letters, I'd never have found these last options.

Changing the _DEFAULTSCREENS option did not seem to affect the number of local consoles.

Working through these problems took a number of hours spread over a week to solve. I cannot find any documentation on the new options. Logically they should not be related. Keyboard functions should not depend on mouse or video functions but empirically there seems little doubt that all these depend on each other. Perhaps a few aren't needed but there is a limit to how many combinations one can try and how long one can wait for a erratic event such as a keyboard lockup.

Previously the multiple virtual consoles worked with no special kernel settings. Now approximately 30K of apparently unrelated devices and options need to be on to have multiple virtual consoles. I noticed man listings were now appearing with limited colors. I've read monochrome man and ls listings for years and have no difficulty finding what I need. I find Linux's technicolor ls listings more of a distraction than a help. I'm trying to make OpenBSD even more secure but it's not worth hours of trial and error effort for at best, limited gains. Perhaps these new options have some real functional benefit that's not documented or documented and I have not seen. If however they are motivated by glitz (color text displays) over OpenBSD's traditional core values of quality and reliability leading to security, then the OpenBSD development team is in danger of losing it's way.

transparent spacer

Top of Page - Site Map

Copyright © 2000 - 2014 by George Shaffer. This material may be distributed only subject to the terms and conditions set forth in (or These terms are subject to change. Distribution is subject to the current terms, or at the choice of the distributor, those in an earlier, digitally signed electronic copy of (or cgi-bin/ from the time of the distribution. Distribution of substantively modified versions of GeodSoft content is prohibited without the explicit written permission of George Shaffer. Distribution of the work or derivatives of the work, in whole or in part, for commercial purposes is prohibited unless prior written permission is obtained from George Shaffer. Distribution in accordance with these terms, for unrestricted and uncompensated public access, non profit, or internal company use is allowed.


What's New
Email address

Copyright © 2000-2014, George Shaffer. Terms and Conditions of Use.