Using the lmdb database backend: Difference between revisions

From SambaWiki
m (Fix broken ext link)
 
(11 intermediate revisions by one other user not shown)
Line 1: Line 1:
= LMDB back end =
= LMDB back end =
Samba can use an [https://symas.com/lmdb/ LMDB] back end instead of the default [https://tdb.samba.org/ TDB] back end. The [https://symas.com/lmdb/ LMDB] back end permits database sizes greater than 4Gb.
Samba can use an [https://symas.com/lmdb/ LMDB] back end instead of the default [https://tdb.samba.org/ TDB] back end. The LMDB back end permits database sizes greater than 4Gb.

== Enabling LMDB ==
The LMDB backend can be enabled when provisioning or joining a domain using the <code>--backend-store=mdb</code> option. The maximum size of the database files defaults to 8Gb, this can be changed with the <code>--backend-store-size</code>

{{Imbox
| type=warning
| text=This is a hard limit and Samba will fail if this limit is exceeded. There is currently no way to resize a Samba LMBD database.
}}

== Notes ==
=== High virtual memory allocation ===
LMDB uses memory mapped files so if using the default database size of 8Gb the virtual memory reported by htop for Samba processes is typically 40Gb or 80Gb. This is normal and is not normally a concern, see the following article on
[https://kb.symas.com/lmdb-database-file-sizes-and-memory-utilization.html lmdb memory utilisation and file sizes].

=== Database file sizes ===


=== Identifying an LMDB file ===
All Samba LMDB data files have an associated lock file with the extension <code>.ldb-lock</code> and the data file has a </code>.ldb</code> extension.

# ls
'CN=CONFIGURATION,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb'
'CN=CONFIGURATION,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb-lock'
'CN=CONFIGURATION,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb-lock-lock'
'CN=SCHEMA,CN=CONFIGURATION,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb'
'CN=SCHEMA,CN=CONFIGURATION,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb-lock'
'DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb'
'DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb-lock'
'DC=DOMAINDNSZONES,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb'
'DC=DOMAINDNSZONES,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb-lock'
'DC=FORESTDNSZONES,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb'
'DC=FORESTDNSZONES,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb-lock'
metadata.tdb

=== Calculating space used ===
As discussed in the article [https://symas.com/understanding-lmdb-database-file-sizes-and-memory-utilization/ lmdb memory utilisation and file sizes], the size of the database on disk does not reflect the actual space used. To calculate space used, use the <code>mdb_stat</code> tool:

# mdb_stat -ne 'CN=CONFIGURATION,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb'
Environment Info
Map address: (nil)
Map size: 8589934592
Page size: 4096
Max pages: 2097152
Number of pages used: 2013
Last transaction ID: 4
Max readers: 100000
Number of readers used: 3
Status of Main DB
Tree depth: 3
Branch pages: 26
Leaf pages: 1267
Overflow pages: 661
Entries: 7315

The space used is <code>Page size * (Branch pages + Leaf pages + Overflow pages + 2)</code>, where 2 is the number of meta/control pages at the start of the file.

Latest revision as of 23:20, 18 October 2023

LMDB back end

Samba can use an LMDB back end instead of the default TDB back end. The LMDB back end permits database sizes greater than 4Gb.

Enabling LMDB

The LMDB backend can be enabled when provisioning or joining a domain using the --backend-store=mdb option. The maximum size of the database files defaults to 8Gb, this can be changed with the --backend-store-size

Notes

High virtual memory allocation

LMDB uses memory mapped files so if using the default database size of 8Gb the virtual memory reported by htop for Samba processes is typically 40Gb or 80Gb. This is normal and is not normally a concern, see the following article on lmdb memory utilisation and file sizes.

Database file sizes

Identifying an LMDB file

All Samba LMDB data files have an associated lock file with the extension .ldb-lock and the data file has a .ldb extension.

# ls
'CN=CONFIGURATION,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb'
'CN=CONFIGURATION,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb-lock'
'CN=CONFIGURATION,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb-lock-lock'
'CN=SCHEMA,CN=CONFIGURATION,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb'
'CN=SCHEMA,CN=CONFIGURATION,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb-lock'
'DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb'
'DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb-lock'
'DC=DOMAINDNSZONES,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb'
'DC=DOMAINDNSZONES,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb-lock'
'DC=FORESTDNSZONES,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb'
'DC=FORESTDNSZONES,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb-lock'
metadata.tdb

Calculating space used

As discussed in the article lmdb memory utilisation and file sizes, the size of the database on disk does not reflect the actual space used. To calculate space used, use the mdb_stat tool:

# mdb_stat -ne 'CN=CONFIGURATION,DC=ADDOM,DC=SAMBA,DC=EXAMPLE,DC=COM.ldb'
Environment Info
  Map address: (nil)
  Map size: 8589934592
  Page size: 4096
  Max pages: 2097152
  Number of pages used: 2013
  Last transaction ID: 4
  Max readers: 100000
  Number of readers used: 3
Status of Main DB
  Tree depth: 3
  Branch pages: 26
  Leaf pages: 1267
  Overflow pages: 661
  Entries: 7315

The space used is Page size * (Branch pages + Leaf pages + Overflow pages + 2), where 2 is the number of meta/control pages at the start of the file.