Comparing DTLS and UDP transport protocols for massive multiplayer online (MMO) games
Written by Stuart Middleton on 12 January 2010   

Performance Evaluation

...........................................................

As part of the Edutain project a series of performance experiments were run to compare secure and insecure transport layer protocols suitable for online gaming. We compared Datagram Transport Layer Security (DTLS) and User Datagram Protocol (UDP) protocols with tests run on both a controlled dedicated networked testbed and on the Edutain real-time framework running an online game. The results have informed us on the performance penalties security protocols have for online games. hunter-locked

...........................................................

Innovations demonstrated:

  • Performance comparison of secure DTLS and insecure UDP protocols for online game usage

Funding project Edutain

Stuart Middleton, Bassem Nasser, Panos Melas

Experiment Design

This experiment was designed to benchmark for massive multiplayer online games secure and insecure transport protocol performance. Clear benchmark performance figures are somewhat hard to come by in the literature today, being performed mostly for specific application domains such as the telecommunication industry.

Our use case was to inform our Edutain end users about online game and eLearning performance overheads to expect when using a secure protocol (DTLS) as opposed to the traditional insecure UDP protocols. Network performance is of critical importance to these application domains so concrete performance comparisons are useful when making business decisions regarding security technology uptake.

We setup both a highly controlled testbed and a realistic game testbed to run our experiments on. The controlled testbed consisted of two identical laptops and a dedicated network link; laptops were run using LINUX in a single user admin mode to reduce unexpected CPU and network activity. The game testbed was setup to run the Edutain real-time framework with an online game application. In both cases performance statistics were measured over many test iterations, results averaged, statistical significance computed and different protocols directly compared.

Results

Controlled testbed experiments

The controlled testbed consisted of a Ethernet link [100MBit ] and 2 laptops [Ubuntu 9.04, 1.7 GHz, 1GByte RAM].

DTLS provides a secure alternative to UDP communication, with a straightforward porting of existing UDP-based applications to DTLS. There is no significant benefit from DTLS compression of small messages on relatively fast networks; compression is only worthwhile for transmission on slow networks where text data is larger than 1 MByte in size. DTLS itself adds about 60 bytes to each message size compared to UDP, making small messages highly inefficient to send.

Message

Network ideal (µs)

Server read overhead (µs)*

Total trip (µs)

Size (bytes)

Ethernet [100M]

UDP

DTLS

UDP

DTLS

1

<1

3

187

3

187

64

1

3

195

4

196

128

10

3

205

13

215

512

40

3

232

43

272

1024

80

4

286

84

366

8192

640

60

658

700

1298

16384

1280

318

1065

1598

2345

* Results showed standard deviations of at worst 20%, and more usually below 5%

The server read overhead is of course dependent on the server hardware. Running the same test on a 3GHz machine reduced the overhead by about 66%. Therefore as server and client hardware improves the DTLS overhead becomes less significant compared to the network hardware transmission time, making it a more viable technology choice.

 

Game testbed experiments

[due to be finished in Feb 2010]

 

Further information can be found on the Edutain web site and papers found on e-prints.

 

e-prints publications

Paper due Feb 2010

 

 

Add comment

Security code
Refresh

 

Meet the People

 
Andrew Tickner
 
Stephen Phillips
Stephen is a senior research engineer and project manager working on the application of service oriented infrastructures to the engineering, e-learning and media domains.
 
Panos Melas
 
Nicola Salvo
 
Sue Jenkins