Paged Results

From SambaWiki
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

For Samba 4.10, the behaviour of the paged results control (1.2.840.113556.1.4.319, RFC2696) has been changed to more closely match Windows servers, to improve memory usage. Paged results may be used internally (or is requested by the user) by LDAP libraries or tools that deal with large result sizes, for example, when listing all the objects in the database.

Previously, results were returned as a snapshot of the database but now, some changes made to the set of results while paging may be reflected in the responses. If strict inter-record consistency is required in answers (which is not possible on Windows with large result sets), consider avoiding the paged results control or alternatively, it might be possible to enforce restrictions using the LDAP filter expression.

Previous behaviour

Paging works by splitting the result set into pages of a certain size, specified in the control. As long as there is a new page, a cookie is returned to the client and the client must provide the cookie in order to continue to search.

Prior to Samba 4.10, the very first search providing a paged results control controlled what

RFC2696 does not specify how an implementation should behave in the case of simultaneous modification. Before Samba 4.10, results were temporarily saved in memory and were fed back the client as required.

New behaviour

Real world example

Although the consistency has been changed, realistically most clients will not see any impact. Paged results normally cause a stream of requests to happen in rapid succession, leaving little time to modify the database in between. Besides full searches, attributes of interest should not be changed all that frequently to begin with and smaller sets of results are not broken across pages either.

There are not that many ways to correlate data across the directory by default, and maintaining this data is quite difficult because of the lack of transactions over LDAP and the requirement for clients to perform constrained LDAP modification consistently in every case. One obvious example of data which refers to multiple records is a linked attribute relationship, like member. Performing paged results while updating group memberships may produce stale memberships (which would have also been the case before 4.10), a mix of old and newer relationships, but also you might find a mismatch in member and memberOf links (and relationships to objects which may not be returned during a paged search because they were removed). Without paging enabled, a single search which returns all the groups and users will have a consistent pairing of member and memberOf. With paging enabled, this consistency might be lost due to concurrent changes. Tools that use paging should be aware of a potentially changing LDAP database underneath.

Quirks and oddities (also in Windows)

Moving an object in the result list

Renaming an object or moving an object to another OU for instance, returns the object with a DN which may not match the original DN subtree scoping. This is a behaviour shared with Windows (and likely their virtual-list-view implementation as well). Modification of an object to no longer match the filter expression is forbidden however.

Making changes to search requests during paging

Modification of controls and controls data

Before Samba 4.10, because only the first search did any work, any subsequent controls and changes to the expression of attribute lists were completely ignored.

Modification of the filter expression

In Samba 4.10,

Modification of the attribute list

raising an exception

Excessive modification of the database

This behaviour only happens under Windows, as Windows appears to rely on some form of underlying database index to implement paged results. If you delete too much, these indexes appear to cause the search to fail with a variety of errors (LDAP_UNAVAILABLE_CRITICAL_EXTENSION, LDAP_OPERATIONS_ERROR).

Modifications sometimes can cause the search to be retriggered and presumably the underlying indexes to be rebuilt.

Differences in behaviour with the sort control

Independent issues

External race conditions

Without using LDAP constrained modifications to ensure no one else has changed a particular value, it is quite easy to introduce a race condition.