Editing files locally on server: interoperability

From SambaWiki

General Information

Samba file server (smbd) implements access to unix files from SMB clients. This means, among other things, that multiple clients can access and modify files at once. SMB file access protocol allows clients to know when a file is modified on server (for example by another client or by the server itself), allows for efficient local file caching as long as the file hasn't been modified on the server, and for file locking, to prevent others from modifying (or reading) the file while it is being used (or written) by one of the clients. This all is to ensure everyone see consistent and up to date data and to prevent unwanted concurrent updates and losing data in the process. It has always been common in windows world to have files locked when a process had a file open, this is ensured by the system kernel by default. The same semantics is extended to network file access too, so if a file is being open from anywhere on the network, it is locked locally and for all clients too. On unix this has not been the case though, — the only case of this automatic file locking is for executables which are being executed (ETXTBUSY error), which is not used much on linux today.

There are several aspects of this concurrent file access.

  • open file locking preventing access or modification of a file being open for writing or read. This is something which is always implemented on a local filesystem by windows.
  • notification of file change on server: when a file is changed on server, clients don't know about this without special setup. SMB protocol includes ways for a client to subscribe to change notifications, and the server will send such notifications on actual file changes. (Windows) clients usually performs such subscription automatically.
  • remote file caching locally: for better efficiency and speed, it is possible for a client to cache some files locally. This is done in forms of file leases (or oplocks). Also usually requested by windows clients automatically.
  • ability to invalidate local caches when the file on server changes. This is to ensure clients do not continue caching stale data after the file changed on the server.

File Locking

When an SMB client opens a file and requests it to be locked, smbd does this by creating records in an internal tdb database, /run/samba/locking.tdb on a modern system. This way, all smbd processes know about currently locked files. Optionally, smbd can duplicate this file locking information using POSIX locking mechanism (fcntl(2) locks), so that non-samba processes can know about these locks. POSIX file locks also works over NFS (in case samba is servicing files which are located on another NFS server). POSIX locks are enabled by default (posix locks configuration parameter).

This is about the situation where unix local process is explicitly aware of possible concurrent modifications of the file by another process which also implements this concurrency awareness. It doesn't work for regular unix file utilities such as cat/tee/etc. POSIX locks are not mandatory like locks on Windows and a process has to explicitly ask for file locks. It is still possible to read, modify, delete or rename locked files on samba server behind samba's back (and behind other processed who utilize the same POSIX file locking mechanism).

Linux had mandatory file locking in the past (flock(LOCK_MAND)), but this feature has been removed as unused, by this commit. Some filesystems still implement this feature (notable GPFS). Samba does not implement this feature directly, but it has vfs_gpfs stackable module which does.

Mandatory locking is done by samba if kernel share modes smb.conf parameter is turned on. It is off by default because linux generally does not implement mandatory file locking.

Basically, there is no way for samba to ensure the same semantics for file locking as provided on windows. Most tools on unix will write/delete/rename files which are locked on windows without noticing the lock existed, including wast majority of various file editors. Only specially written tools might notice locks put by samba on behalf of windows.

File Change Notification

File Alteration Monitor, or FAM. An SMB client subscribes to file change notifications on the server, which allows it to know about file modifications. Done by default by windows clients. Samba server uses inotify on linux to implement this feature, it is fully supported, controlled by kernel change notify smb.conf parameter (enabled by default).

This is where interoperability context comes into play: without extra steps, a unix process does not know about modifications done to files by other unix processes. While smbd might notify other smbds about files it modifies on behalf of its clients, this can't be said about other processes running on the same server doing their own work. For example, a unix user might change a file in their home directory, and this change should become visible to all clients accessing this file or directory at the same time. Smbd enables inotify monitoring for files/directories on behalf of client requests, and such changes becomes immediately visible to SMB clients.

Please note: FAM is not used by samba to break file locks or leases (below). It is used only to implement subscription-based change notification mechanism. It could be used for breaking leases, maybe, but it is not.

Local Caching of Remote Files and Invalidating the Cached Copies

For better speed and efficiency, SMB clients are allowed to cache remote files locally (usually in RAM). This is done in forms of SMB2 file leases or oplocks. The idea is that the client asks the file server for a "lease" for a given file, and if the server grants one, this lets the client to cache the file. By granting a lease, server undertakes to notify the client if the file changes on server, so the cached copy must be re-validated, — so client knows when its cached copy might become obsolete.

More about file leases and oplocks on Microsoft site

This feature is controlled on samba side by oplocks smb.conf parameter, more advanced level2 oplocks, and smb2 leases, all are enabled by default by samba. Internally, samba implements this in a form of a .tdb file, /run/samba/leases.tdb, so all smbd processes running on the server know this information.

When a client is about to change file leased by some clients, samba will send an oplock break request (or equivalent) to the clients which have this file leased, — which might take some time.

However, in order for samba to notice leased file modifications performed by other processes on the server (inter-smbd communcation is implemented internally by samba), another mechanism is used by samba: it is fcntl-based file leases, which works in a way similar to windows/SMB file leases (fcntl(3) manpage, F_SETLEASE). This mechanism is controlled by kernel oplocks smb.conf parameter, and it is not enabled by default, because this mechanism has some tradeoffs.

Once again, by default file leases are not revoked when the file cached by client is modified locally. Specify kernel oplocks = yes to enable this feature.

It is okay for shares where file modifications are only performed via samba, since internally, different smbd processes perform inter-process notifications of leased file changes without the help from the kernel on the server, and this allows samba to work in most efficient way. However if the share is used via samba and the same files can be modified on the server, kernel oplocks should be enabled, or else there's a risk to have various interesting effects. One of possible outcomes of editing a file (for which a lease has been granted by smbd) locally is a windows error "Error 0x80070057: The parameter is incorrect" when attempting to open this file on windows.

By turning kernel oplocks on, smbd disable more advanced SMB2 file caching, including durable handles and level2 oplocks. Also, fcntl-based F_SETLEASE mechanism works in form of unix signals, which might be less efficient and more difficult to implement right than other mechanisms.

Inotify subsystem is not used for this task by samba, only fcntl leases.