Testing against Windows with wintest
wintest is a framework for testing Samba against Windows virtual machines. It aims to provide a reliable, repeatable test system that any Samba developer can setup.
The core of wintest is based on pexpect, a python expect system that provides a flexible way of controlling command line tools.
Some background information on wintest is available in this blog post
Setting up your environment for wintest
To run wintest you need a Linux host that can control Windows VMs. Any scriptable VM system that supports snapshots should work. There are currently four example config files for wintest. If you build a config for a different VM system, please contribute an example config file.
- tridge.conf for using VirtualBox.
- abartlet.conf for using virsh to control KVM.
- zahari-esxi.conf using VMware ESXi via ssh.
- bbaumbach.conf using VirtualBox in headless mode, since some systems have problems running VMs with "su" in default gui mode.
The current test script can use and test the following VMs:
You don't need to setup all of these VMs to run the test. The script will look for what VMs you have defined in your config file and will only run tests against those VMs.
Setting up a VM
Each of the VMs needs to be setup with the following properties:
- the telnet server needs to be enabled
- for VMs that will join Samba as a workstation, netdom.exe needs to be installed
- for VMs that are DCs, dcdiag.exe needs to be installed
- the machine needs to have a snapshot with it fully booted
It is also a good idea to do the following:
- ensure that in the snapshot the machine is activated, and windows update is disabled
- it is a good idea to disable the firewall, although the script will try to do that if telnet is working
Make sure you test that telnet into the machine works. You may need to add "Authenticated Users" to the TelnetClients group.
Running the tests
You need to run the tests as root. After you have created a config file (based on one of the examples), you should run it like this:
sudo wintest/test-s4-howto.py --conf wintest/conf/tridge.conf --rebase
this will run the full set of tests, using the parameters from your config file. See the config file for details on where it gets the source tree from and where it builds it. On my laptop the full test suite currently takes about 30 minutes.
Your may find this sample output useful to see what happens when you run wintest.
Without setting a special DNS backend the test will setup a bind9 with the dlz backend for dynamic updates. In newer versions the Samba internal DNS can be used by running wintest with --dns-backend=SAMBA_INTERNAL.
The tests aim to run Samba in a manner as close to real production use as possible. To make this possible, it will modify your /etc/resolv.conf to point at the DNS server starts. The DNS server config is setup to automatically forward DNS requests for non-wintest hosts to your original nameserver, so it should not impact on normal usage of your machine. The test restores your resolv.conf to its original value when the test is complete.
The script will create an IP alias on whatever network interface you setup in your config file. Samba and the DNS server will be setup to listen on that IP alias, and the Windows VMs will be automatically modified to setup their networking to point at the IP alias.
The idea behind this arrangement is that you can run this script on your primary development workstation (eg. your laptop), and you will be able to keep using the machine for normal work while the test is running.
What is tested?
The main things that are tested are:
- joining WinXP and Windows7 to a Samba4 domain as member servers
- joining Windows2003, Windows2008 and Windows2008R2 as DCs in a Samba domain
- joining Windows2008R2 as a RODC to a Samba domain
- joining Samba as a DC in a Windows2003, Windows2008 and Windows2008R2 domain
- joining Samba as a RODC in a Windows2008R2 domain
- DRS replication between Windows and Samba for all the above DC arrangements
- dynamic DNS with TSIG/GSS, using bind9
- dynamic DNS (until now) without TSIG/GSS, using the internal DNS Server if selected
The list of tests is likely to grow rapidly as wintest is developed further.
There is a skeleton of a Samba3 test in wintest, but it needs to be developed further. It would be great if someone could volunteer to expand it to be a useful test of Samba3 against Windows.
We hope in the future to integrate wintest with the build farm, providing web based access to regular testing of Samba against Windows. The main thing that needs to be done is to convert wintest to use the subunit python testing framework that the build farm uses.