Writing Torture Tests: Difference between revisions

From SambaWiki
No edit summary
Line 40: Line 40:
These assert macros will set the torture_result state and return from the function. Often a test wants to do some cleanup on failure, such as delete test files. In these cases there are special <b>torture_assert_*_goto()</b> macros which will jump to a label instead of returning from a function. Some tests also have their own ''CHECK_*()''' macros which wrap the '''torture_result()''' function. Either of these methods is acceptable.
These assert macros will set the torture_result state and return from the function. Often a test wants to do some cleanup on failure, such as delete test files. In these cases there are special <b>torture_assert_*_goto()</b> macros which will jump to a label instead of returning from a function. Some tests also have their own ''CHECK_*()''' macros which wrap the '''torture_result()''' function. Either of these methods is acceptable.


==Using Test Suites==
==Use Test Suites==

All new tests should be added under a sub-level test suite. For instance when adding a new locking test to the ''raw'' directory it should be added in the lock.c file with an entry in the '''torture_raw_lock()''' initialization function. This allows users of smbtorture to run individual tests such as '''RAW-LOCK-ASYNC'''. New tests <font color="red">should not</font> be added to the upper level suites such as '''RAW''' or '''BASE'''.


=Test Verification=
=Test Verification=

Revision as of 00:23, 4 December 2009

The smbtorture framework provided in Samba is an extremely powerful tool for exercising obscure parts of Windows protocols and determining how different server implementations, Windows among them, respond. Writing torture tests is an invaluable way to determine a server's behavior in odd scenarios. Before the release of the Microsoft protocol documentation smbtorture was the only way to determine how a Windows server would behave to packets that were legal but would not be sent by the existing Windows client.

This page attempts to layout some guidelines for development of new torture tests. Primarily this page focuses on adding tests to the source4/torture collection.

Source Layout

The source4/torture directory contains the following sub-directories:

  • basic - this directory primarily contains the original SMB tests that were ported from the source/torture test directory. These tests use the smbcli_*() wrapper functions which provide an abstracted API to the SMB protocol behavior. These tests primarily deal with older aspects of the SMBv1 protocol and should be considered legacy. If possible, new tests should not be added to this directory.
  • raw - this directory contains SMBv1 tests that exercise most aspects of the file sharing protocol by tweaking the raw bits sent in every packet. Most new SMBv1 correctness tests are added to this directory.
  • smb2 - this directory contains correctness tests for the SMBv2 file sharing protocol. Like the raw directory these tests specify each individual bits in each packet. New SMBv2 tests should be added to this directory.

Best Practices

When adding new tests to smbtorture, it's easiest to copy-n-paste an existing test and modify it to behave differently. However, there are several best practices that should be followed for all newly written tests, even if they go against the existing code structure of that file.

Printing Comments

The printf() function should not be used. Instead the torture_comment() function can be used for printing messages in a torture test. Different front ends to torture, including the Samba build farm, may want to format or ignore text coming from the torture tests. Using the torture_comment function allows for this.

Similarly torture_warning() can be used to display warning messages.

Checking Failure

Older tests simply returned false to indicate that a test failed. Torture now keeps a torture_result state for each test which should be set to one of

enum torture_result {
        TORTURE_OK=0,
        TORTURE_FAIL=1,
        TORTURE_ERROR=2,
        TORTURE_SKIP=3
};

The easiest way to set this state is by using the torture_assert_*() macros provided in lib/torture/torture.h.

These assert macros will set the torture_result state and return from the function. Often a test wants to do some cleanup on failure, such as delete test files. In these cases there are special torture_assert_*_goto() macros which will jump to a label instead of returning from a function. Some tests also have their own CHECK_*()' macros which wrap the torture_result() function. Either of these methods is acceptable.

Use Test Suites

All new tests should be added under a sub-level test suite. For instance when adding a new locking test to the raw directory it should be added in the lock.c file with an entry in the torture_raw_lock() initialization function. This allows users of smbtorture to run individual tests such as RAW-LOCK-ASYNC. New tests should not be added to the upper level suites such as RAW or BASE.

Test Verification

Latest Windows