Traffic replay

Revision as of 00:43, 5 June 2019 by Timbeale (talk | contribs) (added Collaborative testing section)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

The traffic_replay tool generates traffic in order to measure the performance of a Samba DC. It can be used to test how well Samba will scale as the network increases in size. It can simulate multiple different hosts making multiple different types of requests to a DC, and so can be used to determine the point at which the DC becomes overloaded.

The traffic_replay tool was added in Samba v4.8. Refer to its man page for more details on how to run it.

How it works

The traffic_replay tool approximates the traffic seen on a real Samba domain, and allows you to increase the amount of traffic, and the rate that it's sent at, in order to simulate extra stress on the AD DC.

The basic idea is that you take a packet capture of the AD traffic on your network, and use this to produce an anonymized traffic model file. The traffic model summarizes the types of AD traffic on the network, as well as their frequency/probability. The traffic_replay script takes this traffic model and adjusts it to fit the scale/load you're trying to simulate. The information in the traffic-model is generic enough that it can be used on any AD DC.

The traffic_replay script itself is built around the concept of conversations. Each conversation represents a network host talking to the AD DC in some way. The traffic model file represents the current conversations in your network. The traffic_replay script allows you to increase the number of conversations to simulate a larger network. For example, specifying --scale-traffic=2.0 would simulate double the amount of AD traffic on your network.

Once finished, the traffic_replay script outputs a summary of the latencies seen for the various AD traffic types that were sent. This allows you to see how large the network could get before users become impacted by the latency of certain AD operations.

Note that several factors can impact the performance of Samba AD:

  • The total number of users and number/size of groups they are in.
  • The types of AD traffic/operation that occur in your network.
  • The number of DCs in your lab domain (i.e. background DRS replication).
  • The CPU resources and disk speed of your DC under test.

Steps to use the tool

The steps involved in using the tool are:

  • Create a samba lab-domain.
  • Take a packet capture of the AD traffic on your domain, and convert this into an anonymized traffic-model file that can be fed into the traffic_replay script. Refer to traffic_summary.pl --help and the traffic_learner man pages for more details, but in general the command to do this would look something like:
tshark -r capture.pcapng -T pdml | traffic_summary.pl | traffic_learner -o traffic-model.txt
  • Run the traffic_replay script, pointing it at the lab DC (its full DNS-name) and passing in your traffic-model. An example command might look like:
traffic_replay -U Administrator%labAdminPwd01 --realm SAMDOM.EXAMPLE.COM --workgroup SAMDOM --fixed-password s0mePwdForUsersCre@ted --latency-timeout 5.0 --random-seed=1 your-model.txt --scale-traffic=2.0 lab-dc0.SAMDOM.EXAMPLE.COM
Note that:
    • --random-seed is optional and can be anything, but it should help to ensure repeated traffic-runs produce consistent results.
    • --latency-timeout allows some leeway at the end of the test for the last few conversations to complete.
    • --fixed-password is the password given to the user accounts that get created for traffic replay.

User generation

The traffic_replay tool generates user and machine accounts for the traffic it generates. As the user-generation is quite slow, the generated accounts will hang around afterwards, so they can be reused for subsequent traffic_replay commands. The generated accounts are under a 'traffic_replay' OU, and account names are generally 'STGU-0-x' for users, 'STGG-0-x' for groups, and 'STGM-0-x' for machines. Use the --clean-up option to remove these accounts.

Populating a large database

The traffic_replay tool's --generate-users-only provides a convenient way to populate a large database with users and groups. If you want to use the tool for this purpose, here are some tips, applicable for Samba v4.10 onwards:

  • It's much faster to run the command locally on the test DC and pass it the filepath of sam.ldb instead of the server-name. This means the tool writes directly to the DB, avoiding the need to perform every single operation over LDAP. To get the full benefit in speed, use --option="ldb:nosync = true", which skips flushing the memmap after every write. However, only do this on a standalone DC, as otherwise there's potential for DB corruption if other operations (e.g. DRS replication) are happening in the background.
  • Use the --random-seed option. As users are distributed randomly to groups, if you (or someone else) needs to rebuild the DB later, then you'll get the same user-group distribution.
  • Use the samba-tool group stats command to view a summary of how the users were distributed amongst the various groups.
  • Generating users is slow. If you're using a temporary VM, instead of re-generating the users each time you spin up a new VM, you can use the backup tool to create a snapshot backup-file of the database. You can then restore the backup-file on a new VM (instead of doing the samba-tool domain provision step).
If you are using developer testenvs, note that you can load the backup-files into a Customdc testenv.
  • Note that assigning group memberships when adding additional users to an existing DB gets a bit complicated, e.g. going from 10,000 users to 20,000. So it may not distribute the group memberships how you expect - it may be better to simply regenerate the user database from scratch.

Collaborative testing

The traffic_replay tool, and the steps involved to use it, are quite complicated. The Samba AD developers are always interested to learn more about how users are actually deploying Samba. If you are willing to produce and share an anonymized traffic-model file from your network, we may be able to run traffic-replay on a cloud machine and get some results for you. Reach out to the AD developers on the samba-technical mailing-list if you might be interested in doing this.