Copyright Notice

Copyright © 2008 Internet Systems Consortium, Inc. All Rights Reserved.

Abstract

ISC has built a testbed out of Commercial Off-The-Shelf (COTS) servers that can perform full-scale tests of the performance of DNS protocols using real data from TLD and root servers, or synthetic data to test the performance of different mixes. We have been particularly interested in the performance of DNSSEC mechanisms within the DNS protocols and on the use of BIND as a production server for major zones.

High-end name service is typically provided by a battery of 13 servers, any one of which could be selected by the client that received the reply identifying them. Our testbed houses 13 servers and a framework that simulates an adequately-fast internet connection to them, so that our tests mirror as closely as possible the operating environment used by TLD and root name service operators. Test clients issue requests to the servers in the testbed, and outboard measurement systems track the requests and the responses.

We have measured raw query processing speed, DDNS update speed, and the speed of query processing while simultaneously updating via DDNS protocols. We used the COM zone from 5 October 2007 for these tests, and we used a 48-hour trace of the Palo Alto F-Root node as a source of query data.

This work was sponsored in part by the National Science Foundation via grant SCI-0427144.

Table of Contents

1.  Introduction
2.  Identifying appropriate server hardware
3.  Operating system evaluations
4.  Testbed physical construction
5.  Test data stream
6.  DNS Protocol performance measurements
7.  Conclusions

Appendix A. Software availability

1. Introduction

ISC has built a testbed that can perform full-scale tests of the performance of DNS protocols using real data from TLD and root servers, or synthetic data to test the performance of different mixes. We have been particularly interested in the performance of DNSSEC mechanisms within the DNS protocols and on the use of BIND as a production server for major zones.

Ordinary DNS traffic uses UDP datagrams. The protocol standard specifies that the replies be restricted to 512 bytes of payload (RFC1035, #4.2.1). In such a packet, the responding name server can fit 13 answers to a question. If the question asked is 'what are the name servers for domain example.com?', the answer will identify no more than 13 name servers. For this reason, high-end name service is typically provided by a battery of 13 servers, any one of which could be selected by the client that received the reply identifying them.

Our testbed houses 13 servers and a framework that simulates an adequately-fast internet connection to them, so that our tests mirror as closely as possible the operating environment used by TLD and root name service operators. Test clients issue requests to the servers in the testbed, and outboard measurement systems track the requests and the responses.

There probably exist enough supercomputers in the world for us to have built a cluster of 13 supercomputers that would have adequate performance under almost any conditions. But an important purpose of our testbed is to test with affordable commercial off-the-shelf server hardware. There is more value in testing the performance of servers that can be used around the world in DNS operations than in testing the performance of experimental servers that never leave a laboratory. So every component of our testbed has been moderately-priced COTS equipment that realistically can be used by full-scale name service operators.

Although we have tested with both signed and unsigned zones, we did not measure the performance of zone signing or re-signing as part of this project.

1.1. Project outline

The DNS Performance Testing Project was conceptually divided into these phases:

1. Design and document testbed. Identify the proper server hardware through benchmarks and order necessary hardware. Testbed will allow simulation of a universe of 13 TLD servers, using computers of appropriate speed interconnected by a non-blocking switch. Testbed will include enough client computers to present a full simulated load, and will have a control machine and a separate logging machine.
2. Acquire a test request stream by recording 72 hours of F root traffic. Subsets of this test stream will be used for performance measurements.
3. Build testbed. Install and configure the servers and interconnects. Try various operating systems under various configuration settings, including at a minimum FreeBSD, Linux, NetBSD, and Solaris. Get the testbed working with BIND under each such OS using a copy of the live .com database, and measure performance under each.
4. Filter the test request stream to produce a TLD test stream for COM by removing all queries that are not asking about .com domains. Modify the "nsupdate" program so that it can generate and send a synthetic test update stream that is 1/3 NS record changes, 1/3 domain-add requests, and 1/3 domain-delete requests, and so that it has a speed-control option that can be used to adjust the rate at which it sends such requests.
5. Set up the testbed so that each of the 13 servers is running as an authoritative server for a recent copy of the COM zone, and that each is processing the filtered test request stream produced in Task 4. Set up the 13 servers so that each uses Notify and IXFR to keep its COM zone up to date. Set up a 14th server as a "Stealth master" (see RFC1996 #2.1) as an authoritative server for the COM zone, and then make a series of tests that use nsupdate to update this 14th server with the synthetic update stream. There will be 3 tests in this series, corresponding to "low", "medium", and "high" rates of update. A low rate is 1% of COM changing every day; a medium rate is 3% of COM changing every day, and a "high" rate will be the maximum speed at which we are able to make it run.
6. Make the testbed hardware available to approved organizations for research measurement projects. Any significant software developed will be provided as open source.

1.2. Staffing

• David Boggs, PhD: principal investigator
• Brian Reid, PhD: associate investigator and research programmer
• Paul Vixie: associate investigator
• Peter Losher: system administrator

2. Identifying appropriate server hardware

Our experience as authors and maintainers of BIND is that the performance of BIND is limited primarily by the processor and memory performance of the server computer, and by bottlenecks in the operating system. Vendors normally advertise processor speeds, but the performance of a computer's memory subsystem is not readily available. We have identified several candidate server computer configurations based on price and availability and have measured the memory performance (bandwidth, latency, and cache performance) of each.

2.1. Candidate computers

We selected computers from various vendors that were priced such that we might be able to buy 16 of them for about $100K. We also included computers that we know were unacceptably slow, so that we could have a wider spectrum of results from the measurement suites. Candidates are shown in Table 2-1. Not all were seriously considered, but we wanted a spectrum of performance to increase our confidece in our benchmark methodology.

We measured a number of other less-powerful systems, primarily to help us be confident in our measurement methodology. None of them was a realistic candidate to be our server choice.

2.2. Measurement methodology

We began with the memtest86 measurement suite (v3.2), by means of a bootable CD that did not involve the subject computer's installed OS. This program claims to measure transfer rates in megabytes/sec to and from memory and caches. We followed with the calibrator (v0.9e), which measures cache and memory latency and TLB performance, and with lmbench (v3.0a4). Combined results are shown in Table 2-2, separated by CPU maker (Intel or AMD). We were suspicious of the memory bandwidth number for the HP 1 unit, presumably because the memtest86 program is not interfacing properly with its hardware. We investigated further using the STREAM system (v1.0), which shows bandwidth numbers similar to those from memtest86 for all units except HP 1. This increases our confidence that the L1 memtest86 measurement for HP 1 is anomalous.

On the basis of these benchmarks, and of vendor prices, we selected the HP 1 system (DL 140g3) to use for our testbed. We began this benchmark assuming that AMD systems would be faster in our price range, but the data shows otherwise.

3. Operating system evaluations

Having benchmarked the hardware for maximum performance in the kind of load to be offered, we now benchmark various operating systems for their suitability to run BIND 9. The same query stream is sent to the same version of the application software running on several different operating systems, and the processing speed is noted.

3.1. Description

The testbed configuration is as shown in Figure 3-1:

Figure 3-1: Testbed configuration

Using several machines in our testbed (one for each OS), we are running an instance of these operating systems:

• FreeBSD 6.2-RELEASE
• FreeBSD 7.0-CURRENT
• Linux Gentoo 2.6.21
• Linux Fedora Core 2.6.18
• NetBSD 4.0
• OpenBSD 4.1
• Windows 2003 Server
• Windows XP Professional

On each of the server test computers, we installed BIND 9.4.1-P1 and configured it to be authoritative for PT, COM.BR, and NL. We configured BIND to act as an authoritative server, with the following configuration:

options {
    listen-on { 204.152.187.83; };
    check-sibling no;
    recursion no;
    fetch-glue no;
    allow-recursion { none; };
    acache-enable yes;
    max-acache-size 128M;
};

3.2. Test methodology

The overall goal of this project is to determine how fast certain computers that are running certain software can process request streams. The application software (BIND name server daemon) runs as a user process under a Unix or Unix-like operating system. Some computers generate requests, send them to the servers under test, and process the responses; one or more server computers receive those requests, process them, and send back the result.

If the server computer is unable to process requests as fast as they are being sent, its operating system will buffer some unprocessed requests and then (when its input buffer fills up) discard any new arrivals. As BIND finishes a request, it will ask the operating system for the next one. That request will be in the input buffer, and as soon as it is removed from that buffer (thereby making space in the buffer), the operating system is again able to accept new requests. Typical packet buffers in the server computer operating system are able to hold between 50 and 100 packets waiting to be processed. BIND 9 ensures that its host operating system has at least 32KB of receiver buffer space, which will hold about 60 found maximum-size query packets.

We take advantage of this operating system behavior for our performance measurements of the server software. The request-generating computer sends a request and keeps track of the responses that it receives. When more than 20 requests are outstanding (the request has been sent but no reply received) the query-sending software stops sending packets until the server catches up. Since the server input queue can hold 50 to 100 unprocessed requests and the sending computer limits itself to having no more than 20 requests outstanding at one time, no queries will be lost for lack of space.

This algorithm for query flow control will ensure that the server always has a pool of queries waiting to be processed, so that its performance limit can be measured by the rate at which it sends back responses to those test queries. The "queries per second" measurements of a server are measurements of the rate at which it is able to send back responses to queries taken from a pool that is never allowed to empty until the very end of the test.

We began with some preliminary tests to determine whether or not the size of the zone file being served had an effect on server performance. We were not able to measure any difference in server response rates over a 50:1 ratio of zone file size. We believe that if we had used a zone file that was larger than the amount of physical memory on the server, there would have been significant performance degradation. After concluding that zone file size did not matter, we used the .PT zone for further testing because it was not tedious to copy it from machine to machine during server reconfiguration.

We used a test data stream that was originally captured from queries to ns-ext.isc.org. We extracted those queries that enquired of the .PT zone. The fastest server configurations took about an hour to process the entire stream. There were 1.13 million such queries in the test stream. We sent this query stream in turn to each of the test servers using the queryperf program. We measured the length of time that it took each server to answer all of the queries in this query stream, and from that determined the average query response rate delivered by that configuration of hardware, software, and operating system.

3.3. Results

Table 3-1 shows the approximate average number of queries per second processed by BIND 9.4.1-P1 running on the listed operating system using the testbed hardware in the configuration shown in section 3.1 and the methodology described in section 3.2.

To determine whether or not there were performance differences in serving signed zones, we performed these tests using both signed and unsigned zone files. We did not test signed zones under all operating systems; we did this test using FreeBSD 6.2-stable, making the assumption that relative speed differences would be the same across operating systems. We used FreeBSD 6.2-stable for our tests because it had the best overall performance across all of the protocols that we tested, and is widely used by actual production DNS server operators on the public Internet.

Our test version of the unsigned zone file for .PT is 4.8 MB; the signed version of that file is 33.9 MB. In our test stream of 1.13 million queries we found only minor differences (about 10%) in the speed at which BIND 9.4.1-P1 responds to queries served from a signed version or unsigned version of the zone. There were differences in query size and total network traffic, as shown in Table 3-2:

We did not find the lack of speed difference surprising, because all of the complexity and processing related to signed zones is in the preparation of those zone files. Once the zone file has been read into BIND, the serving of requests from it is mostly a matter of searching a data structure rather than performing cryptographic computations.

4. Testbed physical construction

The logical design for ISC's DNS performance testbed is described in ISC technote TN-2006-2. The equipment fits in a single 7-foot 19-inch four-post rack, and requires 3 15-amp 120VAC power circuits. Because the engineers performing tests can be located anywhere in the world, we are using IPMI to administer and control the computers. This requires having a separate Ethernet to carry the IPMI signalling, requiring each server to have two Ethernet connections. Although IPMI was devised and popularized by Intel, several open-source IPMI cwlients are available.

4.1. Description

We mounted the power controllers at the bottom of the rack and the Ethernet switches at the top of the rack. Because IPMI is a 10MBPS protocol, we used a (less expensive) 10/100 switch for the IPMI network, and a nonblocking Gigabit switch for the data transport network. The servers are mounted on slide rails, and use every second slot in the rack for better cooling.

4.2. Photographs

Click on a photograph to pop up a larger view. These photographs were taken 4 April 2007.

DNS performance testbed rack seen from left rear corner. The cardboard boxes at the top hold racking supplies.	Testbed from the back. Lavender wires (left) are to the Gigabit data network; turquoise wires (right) are to the IPMI network. Power lines run inside a channel built in to the right edge of the rack.
Testbed seen from the front. Two of the servers are removed for OS testing. A power controller is visible at the bottom edge of this picture.	Close-up of the front panel of the servers, showing vendor model number (HP ProLiant DL140 G3). Note the air vent above the CD-ROM drive. Such vents are not common in 1U servers, but because of high CPU speeds, these servers need more air flow over the CPU heat sinks than a typical 1U.

5. Test data stream

The purpose of this project is to test the performance of DNS servers under various conditions. This requires a baseline test stream, to which we will make various adjustments. A 48-hour baseline test stream was captured from the F-root server in Palo Alto on 15-17 November 2006.

5.1. Characteristics of baseline stream

We used tcpdump to capture all packets transmitted by the Palo Alto F-root server. UDP name service replies include the question to which they are the reply. The baseline tests will be run by reproducing the questions in this stream, one question per query packet. Other tests will require augmenting these queries with others (to increase the load) and replacing some of these queries with a different mix (to change the load without increasing it).

Figure 5-1 shows the output from this characterization program run over the 15-17 November baseline stream.

Start: Wed, 15 Nov 2006 06:00:00 +0000, duration: 172800.00 seconds. 
414931073 requests (38.8% failed)
 
Avg rate (requests/second) = 2401.2, 95%ile burst = 3011.0, Max burst = 3921.9
414342756 recursive queries (38.7% failed); 171887553 authoritative replies
 
23587274 TCP packets in 7906809 sessions (0 never closed) not analyzed (5.7% of total packets)
 
By type of query in default class:
  251892742/ 60.7%   A                          107157854 failed  (42.5%)
   41508116/ 10.0%   PTR                          6860091 failed  (16.5%)
   34088345/  8.2%   AAAA                         7343865 failed  (21.5%)
   23454402/  5.7%   SOA                         21453572 failed  (91.5%)
   23030654/  5.6%   MX                           6186641 failed  (26.9%)
   20844336/  5.0%   A6                            166356 failed   (0.8%)
    7440925/  1.8%   SRV                          7211917 failed  (96.9%)
    4729932/  1.1%   TXT                          1613960 failed  (34.1%)
    3715926/  0.9%   NS                            537492 failed  (14.5%)
    1902492/  0.5%   ANY                           860207 failed  (45.2%)
    1111659/  0.3%                                1111659 failed (100.0%)
    1020917/  0.2%   CNAME                         193925 failed  (19.0%)
      50697/  0.0%   NAPTR                          39748 failed  (78.4%)
      31239/  0.0%   TXT CHAOS                         50 failed   (0.2%)
      22763/  0.0%   MAILB                          22763 failed (100.0%)
      19929/  0.0%   A HS                           19929 failed (100.0%)
       6004/  0.0%   X25                             5917 failed  (98.6%)
       4225/  0.0%   Type0                            485 failed  (11.5%)
            (click here for the full chart, including the long tail)

 
Class ANY:
      34296/  0.0%                                   4451 failed  (13.0%)
 
Return codes (percentages are of total query count):
  159636174  38.5%  NXDomain 
     529285   0.1%  FormErr 
     504175   0.1%  update 
      79421   0.0%  echo 
      24769   0.0%  Refused 
      22763   0.0%  NotImp 
       2825   0.0%  notify 
       1788   0.0%  stat 
        178   0.0%  ServFail 
         43   0.0%  server, 
         32   0.0%  zoneInit 
         21   0.0%  updataA 
          8   0.0%  updateDA 
          8   0.0%  zoneRef 
          7   0.0%  inv_q 
          7   0.0%  op6 
          6   0.0%  op3 
          5   0.0%  updateM 
          1   0.0%  op7 
          1   0.0%  updateD 
          1   0.0%  updateMA 

5.2. Plot of baseline stream against time

Figure 5-2: ISC baseline DNS test data set (15-17 November 2005)

6. DNS Protocol performance measurements

In earlier phases of this project we measured raw performance of proposed hardware platforms, and application performance of hardware/OS combinations. We used these measurements to select a test server system and we built a testbed using the selected systems as building blocks.

The ultimate purpose of all of this work is to measure the performance of the DNS protocols with the BIND software running on commodity hardware. We spent time optimizing the speed of the server platform in order to ensure that we were getting the best possible results in our DNS protocol measurements.

In this section we describe the DNS protocol tests that we performed using our testbed and the results that we measured.

6.1. Test data

The test stream used is detailed in Section 5 of this report ("Test data stream"). The authoritative name servers were loaded with the COM zone as distributed by Verisign for 5 October 2007. Licensing restrictions prevent us from making that zone file public, but we can provide any suitably licensed research group with a copy of that file if Verisign no longer has a copy.

We wrote a program named "updgen", an update generator, which reads a zone file and produces from it an input file for the "nsupdate" program. Such input files must be created in the context of the zone file to which they are an update, in order to know the names of zones to be deleted and to make sure that added zones are not already there. During the testing various update streams were generated from various subsets of the COM zone. A tar file of the updgen program that generated them is at ftp://ftp.isc.org/isc/dns_perf/updgen.tar

The same data stream was used for tests of signed and unsigned zone service. The queries are the same whether or not the zone is signed; only the response is potentially different. Our test computers took 2 hours of processing to sign this COM zone, and the resulting signed zone file was 2.6 times larger than the unsigned zone file.

6.2. Test conditions

6.2.1. Test systems used

The physical configuration of our testbed is described in ISC Technote ISC-TN-2006-2. There are additional test conditions that we have imposed in order to make the tests as realistic as possible. The logical and network configuration is described at the beginning of the "Operating system evaluations" section.

For some tests that required us to divide the test zone into slices, each of which can fit into physical memory. The slicing technique is widely used by search engines for the same reason; swapping introduces unacceptable performance delays.

A production name server would never be run with a physical RAM size that would cause it to swap. Since the amount of grant money available to buy servers was fixed, and we have no control over the price of commodity hardware, and since the number of servers needed is determined by the protocol, we had no choice but to buy servers with less memory than a single-system test would have required. Budgetary constraints limited our servers to 16GB of RAM. As discussed in Section 6.2.2, 25GB of RAM was required to hold the entire COM zone when signed.

The price of RAM is complex. The servers that we used have 8 RAM slots, and we populated each with a 2GB part. The 4GB parts were about 6 times more expensive than the 2GB parts; it would have cost an additional $60,000 to outfit the test machines with larger memories at the time the testbed was purchased and built. At Spring 2008 prices, using the same supplier that we used to purchase the testbed computers, a server that according to our measurements could handle an unsplit zone file (25GB of application memory) could be purchased for about $8000 US.

After finishing these tests, we believe that it was serendipitous that we were not able to afford enough memory to make a big uniprocessor system handle everything. That constraint forced us to structure the server complex into slices, with each slice handling a part of a large zone. This means that the architecture that we measured is one that is quite scalable. If a zone file grows beyond the memory limit of a single-computer server, there is no recourse but to replace the server. If the zone has been divided into 5 slices, each served on a smaller computer, than it is very simple and economical to add a 6th server and recut the zone into 6 slices.

For the measurements described in Sections 6.3.1 and 6.3.2, we tested numerous operating systems (as noted in those sections). For the measurements described in Section 6.3.3, all computers in the testbed were loaded with identical copies of FreeBSD 7.0-RELEASE, and name server computers were running BIND 9.4.2. In every case all query test data was transmitted using the version of queryperf that was bundled in the BIND 9.4.2 release (see below under "Sending test data").

6.2.2. Test data used

For queries-per-second measurements and DDNS update measurements we found that the COM zone could be divided into 2 slices that did not cause swapping. For full-scale queries-while-updating measurements we found that the COM zone needed to be divided into 3 slices. BIND 9.4.2 under FreeBSD 7.0-RELEASE used 20GB of virtual memory when loaded with the entire unsigned COM zone, and slightly less than 25GB of virtual memory when loaded with the signed COM zone.

The COM zone file that we used for these tests is 5.8 GBytes, unsigned. A signed version of that file is 15 GBytes. Each unsigned half of the zone file is 2.9 GBytes but causes BIND 9.4.2 to occupy 12.2 GBytes of memory before answering any queries.

Query test streams were partitioned to match the slices of the zone file, using a simple awk command analogous to that shown in Example 6-1, which builds the subset of the queryperf file that corresponds to the 2nd slice of a 3-slice partition:

    cat com.queryperf | awk '
    {
        NAME=$1
        QTYPE=$2
        N=split(NAME,BurstNAME,"\.")
        Part2=tolower(BurstNAME[N-1])
        if ((Part2 <= "f") || (Part2 > "n")) {
            next;
        } else {
            print;
        }
    }  
    ' > com.3part.g-n.queryperf

6.2.3. Sending test data

Server performance tests were executed by using the queryperf program (part of the BIND 9 distribution) on a machine dedicated to sending test data. We determined that a single queryperf process on this hardware base can send 46,000 queries per second under FreeBSD 7.0-RELEASE, and that if there are two queryperf processes running (either on one sender computer or two) that they can, combined, send 60,000 queries per second. Further addition of additional queryperf processes did not increase the test rate beyond 60,000 queries per second emitted from that server. In other words, a single load generator in our testbed cannot test at a rate faster than 60K queries per second in our configuration. We have two load generator computers, enabling a query test rate of 120,000 queries per second. We can, if necessary, use the "DNS Tender" (see Section 3.1) as an additional source of load, thereby providing a maximum query test rate of 180,000 queries per second.

Note that in our testbed the load-sending computers have 4 CPU cores. When one queryperf process is sending at its maximum speed of 46,000 queries per second, the "top" system command indicates 5% idle time and a weighted CPU usage (WCPU) of 86%. With two queryperf processes running, idle time on the 4-processor system drops to 0.2% idle and the weighted CPU usage is 91%. Adding additional queryperf processes beyond these do not change the numbers from the 2-process measurements. This causes us to believe that this is an OS limit, but we did not investigate the reason for that limit.

6.2.4. Note

Performance of zone signing and re-signing is not part of this study. Current trends (2008) are towards hardware accelerator cards and incremental re-signing. We measured neither, except incidentally, during which we noted that our hardware platform was able to sign the entire COM zone (the edition that we are using) in a bit more than 2 hours, and that the signed zone files are about 2.6 times bigger than the unsigned files.

6.3. Test results

6.3.1. Simple queries of the COM zone

To fit into the physical memory available on our test servers, the COM zone was divided into two slices, each containing half of the zone. We ran the tests using BIND 9.4.1-P1 on several different OS configurations, all using the same hardware with the same BIOS settings. Table 6-1 shows the results. Numbers are the number of queries per second that each server computer was able to process at maximum load, averaged over 5 runs with the same data.

Since these measurements are per-server reply rates, the overal testbed is capable of responding to 1 million queries per second using our hardware and software configuration. At that rate it will consume about 4 Gbits/second of outbound network bandwidth, which means that the network speed would be the limiting factor of pure query speed for a production system built using this design.

6.3.2. Updates of the master server

The next measurement was to feed 10,000 DDNS update requests to the master server as fast as it would process them, measuring how long that took. We ran these tests using BIND 9.4.1-P1 on several different OS configurations, all using the same hardware with the same BIOS settings. Results are shown in Table 6-2.

The anomalous behavior of FreeBSD-6.2-Release is, we believe, an error in the FreeBSD autoconfiguration mechanism that caused BIND to be compiled without the file-system commits (fsync) that are required by RFC2136. DNS updates are required to be committed to nonvolatile storage before the response is returned (RFC2136 section 3.5). The underlying hardware is not capable of that many committed file system writes per second, so the FreeBSD-6.2-Release tests could not have been committed to nonvolatile storage. We have no explanation for the anomalously slow performance under Linux-Fedora.

6.3.3. Queries of authoritative servers while they are being updated

The final test was to combine the previous two, and measure the rate at which an authoritative server can respond to queries while simultaneously being updated via IXFR from a master host which is itself under continuous update at a controlled rate. As noted above under "Test conditions", these tests were made with the COM zone divided into thirds. Results are shown in Table 6-3.

We found that the test version of BIND that we were using (9.4.2) periodically paused to write safety copies of the updated zone information to stable storage. In our configuration that is not necessary because the production servers do not have the master copy of the zone information. We made a minor modification to BIND 9.4.2 so that it did not perform such writes, and with that modification in place, it ran nearly as fast when serving queries from a zone under constant update as it did when serving queries from a static zone. The modifications are available as a patch from ISC (contact info@isc.org) and might be included in a future version of BIND if there is general interest.

As described in Section 6.2.1, all of these measurements were made with FreeBSD-7.0-RELEASE.

When we first proposed this work, we assumed that the query processing rate would drop substantially as the update rate increased. At the update rates we had proposed to measure (1% of the zone per day and 3% of the zone per day) the results were not interesting. So we used an exponential growth rate, measuring up to 256 updates per second, which is about 27% of the zone file size.

7. Conclusions

These results are encouraging, because they show that open-source software and commodity hardware are up to the task of providing full-scale production name service on the largest zones in existence. There are several specific conclusions that can be drawn from them.

• Commodity hardware with BIND 9 software is fast enough by more than a factor of 10 to run large production name service at the rates seen by existing large sites, provided that it has enough physical memory to hold the zone data. The fastest burst query rate we noted in the 48-hour sampling interval was under 4000 queries per second (See Figure 5-1), and the worst-case query response rate that we measured was 50,800 replies per second (See Table 6-3). As noted in Section 6.2.1, a commodity computer able to process today's largest zones at full production rates can be purchased for about $8000 at early 2008 prices.
• The rate at which the COM zone experienced changes during 2006-2008 (typically 2-4 updates per second, as reported by dailychanges.com) is about 10% of the update rate at which we started to measure a dropoff in performance of more than 10% (see Table 6-3). Zone update rates vary from day to day, but never during this two-year period was there a day in which the number of updates was more than 10 per second. Our servers showed only a 13% degradation of capacity for responding to queries when there was an update load 25 times as great as that maximum.
• With minor changes to BIND 9 software (as noted in Section 6.3.3), running on a computer with 64GB of RAM that costs less than $10,000, it has the capacity to reply to queries of signed zones at a rate at least 10 times greater than the burst maximum query rates seen in our test data.
• The query processing rate for signed zones is only 10% to 15% slower than the query processing rate for unsigned zones, but it requires more physical memory to serve queries from a signed zone because they contain more information. Although a signed zone file is about 2.5 to 3 times larger than the unsigned version of that same file, the virtual memory needed to hold that file for processing is only about 20% to 25% greater. 32GB of RAM is enough for BIND 9 to hold a signed version of the COM zone without resorting to swapping.

Our testbed is available to qualified DNS and protocol researchers who wish to repeat our experiments or run other experiments; contact info@isc.org to make arrangements.

Appendix A. Software availability

Very little custom software was necessary for this project. There are two packages developed for it:

• A rate-limited version of the Unix "cat" program that is used to control the rate at which updates are fed to name servers. It has been tested and found to be quite accurate over an extremely wide range of speeds, from one update per hour to tens of thousands of updates per second. It is in ftp://ftp.isc.org/isc/dns_perf/rlcat.tar
• A simple script to generate name-server update test streams from zone files. In order to have repeatable tests, we need to have update test streams. Updates include both adds and drops; a record needs to be in the zone before it can be dropped, and it ought not be in the zone if it is to be added as part of the test. For this reason, the test stream must be generated by processing of the zone that will be updated. The simple scripts that we used to do this are in ftp://ftp.isc.org/isc/dns_perf/updgen.tar

Our compilation and coding work was done on a FreeBSD 6.2 system, and these two packages are known to compile on FreeBSD 6 and 7 systems.

Author's Address