The customdc is a special testenv that's only used for manual testing, rather than the automated tests most testenvs are primarily used for. Customdc allows you to specify any backup-file to load into the new testenv.
To run the testenv, you need to specify a 'BACKUP_FILE' shell variable, e.g.
BACKUP_FILE=/tmp/samba-backup-50k-dc-0-mdb-50k-offline.tar.bz2 SELFTEST_TESTENV=customdc make testenv
The main use-case for the customdc is testing changes against a large database. Adding users is very time-consuming, so it's much quicker to populate a domain with users once, take a backup, and then you can spin up a testenv based on the backup multiple times.
Another use-case is that if you get a database that's corrupted or in a bad state, then you could save a backup and be able to easily get the database back into the bad state. This allows you to try different commands to diagnose/fix the issue, without fear of never seeing the problem again.
You could even spin up a 'lab DC' inside a testenv, by taking a backup of a real network DC.