Code Coverage

From SambaWiki

https://samba-team.gitlab.io/devel/samba/

Total coverage

Last updated: 2019-05-14 17:41:12

Line coverage: 39.8% (519470 / 1304847)
Function coverage: 42.0% (30933 / 73567)

Function coverage is generally better than line coverage across the code base. In a number of cases where function coverage is not 100%, the calls are stubbed out and return errors such as 'Not Implemented'. Much of the test code has high levels of code coverage indicating that these tests are actually being run correctly. From looking at what is being covered, it does not seem like there is much trivial testing that has been obviously missed and much of it requires a more in-depth investigation.

Line coverage exceeding 75% normally indicates reasonably good coverage in a file. Reaching higher than this (and 100%) is often impossible without heavy instrumentation because of the nature of C code. When inspecting these files, you can notice that many of the allocation errors and general error cases are not well covered. Under normal operation, and even in extreme error conditions, many of these cases are practically impossible to hit. In some cases, as a result of defensive coding, these cases are in fact impossible to hit (but allow refactoring and other unexpected external changes to preserve the correct behaviour).

Line coverage below 60% may be in need of some further testing, although do note that in some cases, these files have more error cases or difficult to test scenarios and so 60% (or below) may already indicate reasonable coverage.

Notable areas

bin/default/librpc/gen_ndr (Generated Code)

Line coverage: 18.1% (104224 / 574711)

Generated code contributes to almost half of all detected C code. This code is generated using PIDL which reads IDL files and generates (network) marshalling code. In particular, it writes binary serialization and deserialization functions for structures sent over network protocols, formatted print routines for structures and in some cases generates remote procedure call (RPC) interfaces. These RPC interfaces are generally for DCE/RPC protocols implemented by Active Directory and internal messaging. They are often stubbed and allow manual implementation elsewhere in the code.

In many cases, there are no callers of our formatted print routines for many structures. Many structures remain unused, and many structures are deprecated (e.g. v3, v5 are from earlier versions of the protocol and v8 is commonly in use), as well as many functions defined in various RPC protocol specifications. In fact, many exposed functions which are at the latest version may still be superseded by non-RPC mechanisms, or clients generally perform these operations over a different protocol e.g. LDAP.

One other caveat with the generated code is that there is a significant amount of dead-code generated because PIDL is quite simple and may duplicate a number of different checks that LCOV might be detecting. Regardless, the coverage statistics for this set of code is very low and any ways we can improve this warrant investigation.

source4/heimdal (Kerberos Implementation Forked from Heimdal)

This code implements our Kerberos Key Distribution Center (KDC) and includes a number of client libraries and crypto. Although this forms a critical component of the Samba Active Directory implementation, a number of the components in the Heimdal code base which we have copied into our source tree are not used (source4/heimdal/lib/ntlm). Similarly, any tests that come with Heimdal and could be exercising different functionality are not being run within our code. Due to the way we wrap the KDC and our own independent implementation of different security layers (GENSEC and GSSAPI), many paths in this code are never used and should never be used.

source4/dsdb/samdb/ldb_modules

Line coverage: 73.0% (16673 / 22843)

Much of the LDAP server seems to be similarly tested. Underlying backend, ldb_tdb and ldb_mdb + lib/ldb and lib/tdb.

Note there is a skel.c lib/ldb/modules

count_attrs, OpenLDAP backend.

source4/torture and source3/torture (Test Code)

These describe a number of tests written in C code (as opposed to our Python tests), our primary testing method in the past but sometimes written due to the difficulty of creating external bindings. The coverage of the source4 code is notably better than the source3. This warrants more investigation.

source3/torture

Line coverage: 47.1% (6519 / 13830)
Function coverage: 58.1% (293 / 504)

source4/torture

Varies by directory, generally 80%+ line coverage besides nbench.

source3/rpc_server, source4/rpc_server and source3/rpcclient (DCE/RPC)

source3/rpcclient

Line coverage: 6.2% (494 / 8025)


dnsserver (browser has stub functions)

source4/ntvfs (File Server Not Running in Production)

Not well tested, with less than 50% coverage in parts.

source3/modules (Samba File Server VFS Modules)

Line coverage: 37.7% (9698 / 25692)
Function coverage: 47.5% (868 / 1828)

source3/winbindd

Line coverage: 33.8% (6677 / 19765)
Function coverage: 51.2% (511 / 998)

The AD winbind

source3/smbd

Line coverage: 63.4% (26729 / 42164)
Function coverage: 82.3% (1346 / 1636)

source3/nmbd

Line coverage: 38.0% (2175 / 5728)
Function coverage: 53.4% (164 / 307)

third_party

Areas of code to consider removing or consolidating

Group policy code

Registry code