Difference between revisions of "Messaging"

(correct statistics)
(new test results)
Line 42: Line 42:
 
== Testing and benchmarking ==
 
== Testing and benchmarking ==
  
Messaging speed can be benchmarked with LOCAL-MESSAGING test from [[Samba4]] [[smbtorture]]. There is a backport of this test to samba-3 (amd trunk) messaging code - torture/msgtest.c (see current branches).
+
Messaging speed can be benchmarked with LOCAL-MESSAGING test from [[Samba4]] [[smbtorture]]. There is a backport of this test to samba-3 (and trunk) messaging code - ''torture/msgtest.c'' (see current svn branches).
  
Implementations stated above were compared with this tool (-O2 compilation, Thinkpad T41 1700 MHz hardware)
+
Implementations stated above were compared with this tool (average results based on 5 tests, -O2 compilation, Thinkpad T41 1700 MHz hardware):
  
 
* TDB ~ 64850 msg/s (100%)
 
* TDB ~ 64850 msg/s (100%)
Line 55: Line 55:
 
* datagram sockets ~ 7.26 Mb/s (95%)
 
* datagram sockets ~ 7.26 Mb/s (95%)
 
* stream sockets ~ 6.78 Mb/s (90%)
 
* stream sockets ~ 6.78 Mb/s (90%)
 +
 +
and with 5 clients:
 +
 +
* TDB ~ 8.3 Mb/s (100%)
 +
* datagram sockets ~ 7.8 Mb/s (94%)
 +
* stream sockets ~  7.3 Mb/s (88%)
  
 
All results are very rough, but they can be used for a compare.
 
All results are very rough, but they can be used for a compare.

Revision as of 10:40, 23 May 2006

Multiple smbd (and nmbd, winbindd) processes usually run simultaneously - a main daemon process and couple of connections processes. Samba code has special API that allows different processes send messages each other. There are two implementations of the messaging code - different in current Samba3 and Samba4 code.

TDB implementation (samba3 and trunk)

This implementaion uses TDB file for storing messages and UNIX signals for notifying message receiver.

All messages are stored in messages.tdb file (one can find it in var/locks directory). Each message is just a database record that has six fields:

  • messaging version (there is only one version now - 1)
  • message type
  • sender process id
  • receiver process id
  • message data length
  • message data

There are lot of message types in Samba3, you can see them all in include/messages.h file.

Process sends a message to other process with special function message_send_pid and to all processes with message_send_all. Each process should call message_dispatch routinely to check if messages were received and run appropriate handler.

Messaging-tdb.png

(you can get image source here - [1])

UNIX datagram sockets implementation (samba4)

This implementation uses UNIX domain sockets. Each smbd listens on a unix domain (datagram) socket with the name based on its pid. All sockets are located in var/locks directory. One message is packed into one datagram, so now we have message size problem - there is no way to send messages with size larger than allowed by a datagram.

Messaging-socket.png

You can get the patch with this messaging approach for samba3 - [2]

This implementation depends on net.unix.max_dgram_qlen systcl variable - it's the maximum datagram queue length. You should increase the variable's value if your messaging under high pressure (e.g. if testing it with torture tests).

UNIX stream sockets implementation

This implementation uses UNIX domain sockets too, but stream sockets. Each smbd listens on a unix domain (stream) socket with the name based on its pid. All sockets are located in var/locks directory. When a message is about to be sent, the sender connect(2)s to that socket and sends the whole message. As stream sockets are bidirectional, the response can use the same socket. This could be implemented by a message socket cache that holds for example the last N (e.g. 10) connections. All data transfering is being done asynchronousy with main a select loop (message_dispatch does all connections-related work).

Messaging-stream-socket.png

You can get the patch with this messaging approach for samba3 - [3]

Testing and benchmarking

Messaging speed can be benchmarked with LOCAL-MESSAGING test from Samba4 smbtorture. There is a backport of this test to samba-3 (and trunk) messaging code - torture/msgtest.c (see current svn branches).

Implementations stated above were compared with this tool (average results based on 5 tests, -O2 compilation, Thinkpad T41 1700 MHz hardware):

  • TDB ~ 64850 msg/s (100%)
  • datagram sockets ~ 58400 msg/s (90%)
  • stream sockets ~ 16850 msg/s (26%)

Overall bechmarking can be done with dbench tool. Here are results of running dbench with 10 clients:

  • TDB ~ 7.65 Mb/s (100%)
  • datagram sockets ~ 7.26 Mb/s (95%)
  • stream sockets ~ 6.78 Mb/s (90%)

and with 5 clients:

  • TDB ~ 8.3 Mb/s (100%)
  • datagram sockets ~ 7.8 Mb/s (94%)
  • stream sockets ~ 7.3 Mb/s (88%)

All results are very rough, but they can be used for a compare.