The customdc is a special testenv that's only used for manual testing, rather than the automated tests most testenvs are primarily used for. Customdc essentially allows you to copy any Samba database and load it into a testenv.
This feature is primarily aimed at developers. It allows you to save particular Samba configurations or databases, and easily reload them later.
We customize the database that the testenv uses via a backup-file. For more information on backup-files, see Back up and Restoring a Samba AD DC. Basically, the easiest way to generate one is:
bin/samba-tool domain backup offline --targetdir=$HOME/samba-backups --configfile=./st/$ENVNAME/etc/smb.conf
(The $ENVNAME above assumes you're in another testenv, but you can run the same command on a running DC just by changing the --configfile option)
To run the testenv, you need to specify a 'BACKUP_FILE' shell variable, e.g.
BACKUP_FILE=$HOME/samba-backups/samba-backup-50k-dc-0-mdb-50k-offline.tar.bz2 SELFTEST_TESTENV=customdc make testenv
The main use-case for the customdc is testing changes against a large database. Adding users is very time-consuming, so it's much quicker to populate a domain with users once, take a backup, and then you can spin up a testenv based on the backup multiple times.
Another use-case is that if you get a database that's corrupted or in a bad state, then you could save a backup and be able to easily get the database back into the bad state. This allows you to try different commands to diagnose/fix the issue, without fear of never seeing the problem again.
You could even spin up a 'lab DC' inside a testenv, by taking a backup of a real network DC.