Internet Systems Consortium
DNS Performance Testing project

Server Hardware Evaluation

 

Summary

Our experience as authors of the BIND software is that the performance of BIND is limited primarily by the processor and memory performance of the server computer. Vendors normally advertise processor speeds, but the performance of a computer's memory subsystem is not readily available. We have identified several candidate server computer configurations based on price and availability and have measured the memory performance (bandwidth, latency, and cache performance) of each.

Candidate computers

We selected computers from various vendors that were priced such that we might be able to buy 16 of them for about $100K. We also included computers that we know were unacceptably slow, so that we could have a wider spectrum of results from the measurement suites. This included:

Computers tested
ID Vendor Model Code Processor(s) Memory Bus Comments
Sun 1 Sun Sun Fire X4200 4x AMD254/2.8GHz DDR-400 PC3200  
Sun 2 Sun Sun Fire X4200 4x AMD285/2.6GHz DDR-400 PC3200  
Iron 1 Iron Systems I-class I1525HS 2x Xeon/2.4GHz DDR-400 PC3200 Intel E7505 chipset
Iron 2 Iron Systems M-class MS3520HS 4x AMD280/2.4GHz DDR-400 PC3200 Tyan S3870
Cel 1 Celestica   A8440 2x AMD846/2.0GHz DDR-333 PC2700 desktop
DEC Compaq     1x Celeron/1.25GHz SD-133 PC133 desktop
HP 1 Hewlett Packard Proliant DL 140 4x Xeon/3.0GHz DDR2-667 PC5400 1U system

We measured a number of other less-powerful systems, primarily to help us be confident in our measurement methodology. None of them was a realistic candidate to be our server choice.

Measurement methodology

We began with the memtest86 measurement suite (v3.2), by means of a bootable CD that did not involve the subject computer's installed OS. This program claims to measure transfer rates in megabytes/sec to and from memory and caches. We followed with the calibrator (v0.9e), which measures cache and memory latency and TLB performance, and with lmbench (v3.0a4). Combined results are shown below, separated by CPU maker (Intel or AMD). We were suspicious of the memory bandwidth number for the HP 1 unit, presumably because the memtest86 program is not interfacing properly with its hardware. We investigated further using the STREAM system (v1.0), which shows bandwidth numbers similar to those from memtest86 for all units except HP 1.

AMD processors
Intel processors
   
Sun 1
Sun 2
Iron 2
Cel 1
HP 1
Iron 1
DEC
Bandwidth in MBytes/sec
  L1 memtest86
22886
21251
19717
16331
49058
19607
12375
  L2 memtest86
5686
5280
4899
4230
--
14675
5610
  Memory mt86
2237
2202
2237
972
19559
1460
342
  lmbench
2316
2368
1303
2984
2047
652
Latency in nanoseconds (miss/replace)
  L1 calibrator
3.48/3.47
3.73/3.73
4.08/4.07
5.07/5.05
3.07/3.80
6.82/6.94
4/4
  L2 calibrator
99/99
98/98
116/116
165/165
64/104
114/116
102/102
  lmbench
83
83
155
72
109
109
STREAM benchmarks in MBytes/sec
  Copy
1724
1816
1122
2586
1309
285
  Scale
1785
1885
1175
2751
1194
310
  Add
1896
1994
1254
2884
1329
394
  Triad
1893
1958
1138
2890
1524
397
           

On the basis of these benchmarks, and of vendor prices, we selected the HP 1 system (DL 140) to use for our testbed. We began this benchmark assuming that AMD systems would be faster in our price range, but the data shows otherwise.