Internet Systems Consortium
DNS Performance Testing project
Operating system evaluations

Summary

Having benchmarked the hardware for maximum performance in the kind of load to be offered, we now benchmark various operating systems for their suitability to run BIND 9. The same query stream is sent to the same version of the application software running on several different operating systems, and the processing speed is noted.

Description

The testbed configuration is as follows:

Using several machines in our testbed (one for each OS), we are running an instance of these operating systems:

• FreeBSD 6.2-RELEASE
• FreeBSD 7.0-CURRENT
• Linux Gentoo 2.6.21
• Linux Fedora Core 2.6.18
• NetBSD 4.0
• OpenBSD 4.1
• Windows 2003 Server
• Windows XP Professional

On each of the server test computers, we installed BIND 9.4.1-P1 and configured it to be authoritative for PT, COM.BR, and NL. We configured BIND to act as an authoritative server, with the following configuration:

options {
    listen-on { 204.152.187.83; };
    check-sibling no;
    recursion no;
    fetch-glue no;
    allow-recursion { none; };
    acache-enable yes;
    max-acache-size 128M;
};

Test methodology

We began with some preliminary tests to determine whether or not the size of the zone file being served had an effect on server performance. We were not able to measure any difference in server response rates over a 50:1 ratio of zone file size. We believe that if we had used a zone file that was larger than the amount of physical memory on the server, there would have been significant performance degradation. After concluding that zone file size did not matter, we used the .PT zone for further testing because it was not tedious to copy it from machine to machine during server reconfiguration.

We used a 1-hour test data stream that was captured from queries to ns-ext.isc.org. We extracted those queries that enquired of the .PT zone. There were 1.13 million such queries in the test stream. We sent this query stream in turn to each of the test servers using the queryperf program. We measured the length of time that it took each server to answer all of the queries in this query stream, and from that determined the average query response rate at which the server breaks down and is no longer able to process further requests, for each operating system.

Results

This table shows the approximate average number of queries per second processed by BIND 9.4.1-P1 running on the listed operating system using the testbed hardware in the configuration shown at the beginning of this page. The query rate shown is the 'breaking point' capacity such that the server would be forced to drop queries if it received them any faster (on average).

To determine whether or not there were performance differences in serving signed zones, we performed these tests using both signed and unsigned zone files. We did not test signed zones under all operating systems; we did this test using FreeBSD 6.2, making the assumption that relative speed differences would be the same across operating systems. Our test version of the unsigned zone file for .PT is 4.8 MB; the signed version of that file is 33.9 MB. In our test stream of 1.13 million queries we found no difference in the speed at which BIND 9.4.1-P1 responds to queries served from a signed version or unsigned version of the zone. There were differences in query size and total network traffic:

At the server speeds we are measuring ("Results" table, above) maximum-rate server replies from an unsigned zone generate about 5 megabits per second of network traffic, and 13 megabits per second from a signed zone.

Conclusion

We will use Linux Gentoo 2.6.20.7 for further production testing. We brought these numbers to the attention of the FreeBSD development team, and will re-test when FreeBSD 7.1 is released.