Difference between revisions of "The New VFS"

(Fallback on systems without O_PATH support)
 
(2 intermediate revisions by the same user not shown)
Line 5: Line 5:
 
Starting with version 4.14 Samba provides core infrastructure code that allows
 
Starting with version 4.14 Samba provides core infrastructure code that allows
 
basing all access to the server's filesystem on file handles and not on
 
basing all access to the server's filesystem on file handles and not on
paths. An example of this is using =fstat()= instead of =stat()=, or
+
paths. An example of this is using ''fstat()'' instead of ''stat()'', or
=SMB_VFS_FSTAT()= instead of =SMB_VFS_STAT()= in Samba parlance.
+
''SMB_VFS_FSTAT()'' instead of ''SMB_VFS_STAT()'' in Samba parlance.
  
 
Historically Samba's fileserver code had to deal a lot with processing path
 
Historically Samba's fileserver code had to deal a lot with processing path
Line 21: Line 21:
 
file data.
 
file data.
  
Samba's internal file handle structure is of type =struct files_struct= and all
+
Samba's internal file handle structure is of type ''struct files_struct'' and all
variable pointing to objects of such type are typically called =fsp=. Until very
+
variable pointing to objects of such type are typically called ''fsp''. Until very
 
recently the only function that would open such a file handle and return an fsp
 
recently the only function that would open such a file handle and return an fsp
was =SMB_VFS_CREATE_FILE()=.
+
was ''SMB_VFS_CREATE_FILE()''.
  
Internally =SMB_VFS_CREATE_FILE()= consisted of processing through Samba's VFS
+
Internally ''SMB_VFS_CREATE_FILE()'' consisted of processing through Samba's VFS
 
open function to open the low level file and then going through Samba's Windows
 
open function to open the low level file and then going through Samba's Windows
 
NTFS emulation code.
 
NTFS emulation code.
  
The key point of the new helper function which is called =openat_pathref_fsp()=
+
The key point of the new helper function which is called ''openat_pathref_fsp()''
 
is that it skips the NTFS emulation logic. Additionally, the handle is
 
is that it skips the NTFS emulation logic. Additionally, the handle is
 
restricted internally to be only usable as a path reference but not for any sort
 
restricted internally to be only usable as a path reference but not for any sort
of IO. On Linux this is achieved by using the =O_PATH= =open()= flag, on systems
+
of IO. On Linux this is achieved by using the ''O_PATH'' ''open()'' flag, on systems
without =O_PATH= support other mechanisms are used described in more detail
+
without ''O_PATH'' support other mechanisms are used described in more detail
 
below.
 
below.
  
 
Path processing in Samba typically means processing client supplied paths by
 
Path processing in Samba typically means processing client supplied paths by
Samba's core path processing function =filename_convert()= which returs a
+
Samba's core path processing function ''filename_convert()'' which returs a
pointer to an object of type =struct smb_filename=. Pointers to such objects are
+
pointer to an object of type ''struct smb_filename''. Pointers to such objects are
 
then passed around, often passing many layers of code.
 
then passed around, often passing many layers of code.
  
By attaching an =fsp= file handle returned from =openat_pathref_fsp()= to all
+
By attaching an ''fsp'' file handle returned from ''openat_pathref_fsp()'' to all
=struct smb_filename= objects returned from =filename_convert()=, the whole
+
''struct smb_filename'' objects returned from ''filename_convert()'', the whole
 
infrastructure code has immediate access to a file handle and so the large
 
infrastructure code has immediate access to a file handle and so the large
 
infrastructure codebase can be converted to use handle based VFS functions
 
infrastructure codebase can be converted to use handle based VFS functions
Line 50: Line 50:
 
= Samba and O_PATH =
 
= Samba and O_PATH =
 
== Background ==
 
== Background ==
On Linux the =O_PATH= flag to =open()= can be used to open a filehandle on a file or directory with interesting properties: [fn:manpage]
+
On Linux the ''O_PATH'' flag to ''open()'' can be used to open a filehandle on a file or directory with interesting properties: [fn:manpage]
  
- the file-handle indicates a location in the filesystem tree,
+
* the file-handle indicates a location in the filesystem tree,
- no permission checks are done by the kernel on the filesystem object and
+
* no permission checks are done by the kernel on the filesystem object and
- only operations that act purely at the file descriptor level are allowed.
+
* only operations that act purely at the file descriptor level are allowed.
  
The file itself is not opened, and other file operations (e.g., ~read(2)~, ~write(2)~, ~fchmod(2)~, ~fchown(2)~, ~fgetxattr(2)~, ~ioctl(2)~, ~mmap(2)~) fail with the error ~EBADF~.
+
The file itself is not opened, and other file operations (e.g., ''read(2)'', ''write(2)'', ''fchmod(2)'', ''fchown(2)'', ''fgetxattr(2)'', ''ioctl(2)'', ''mmap(2)'') fail with the error ''EBADF''.
  
 
The following subset of operations that is relevant to Samba is allowed:
 
The following subset of operations that is relevant to Samba is allowed:
  
- ~close(2)~,
+
* ''close(2)'',
- ~fchdir(2)~, if the file descriptor refers to a directory,
+
* ''fchdir(2)'', if the file descriptor refers to a directory,
- ~fstat(2)~,
+
* ''fstat(2)'',
- ~fstatfs(2)~ and
+
* ''fstatfs(2)'' and
- passing the file descriptor as the dirfd argument of ~openat()~ and the other "*at()" system calls. This includes ~linkat(2)~ with AT_EMPTY_PATH (or via procfs using AT_SYMLINK_FOLLOW) even if the file is not a directory.
+
* passing the file descriptor as the dirfd argument of ''openat()'' and the other "*at()" system calls. This includes ''linkat(2)'' with AT_EMPTY_PATH (or via procfs using AT_SYMLINK_FOLLOW) even if the file is not a directory.
  
Opening a file or directory with the ~O_PATH~ flag requires no permissions on the object itself (but does require execute permission on the directories in the path prefix). By contrast, obtaining a reference to a filesystem object by opening it with the ~O_RDONLY~ flag requires that the caller have read permission on the object, even when the subsequent operation (e.g., ~fchdir(2)~, ~fstat(2)~) does not require read permission on the object.
+
Opening a file or directory with the ''O_PATH'' flag requires no permissions on the object itself (but does require execute permission on the directories in the path prefix). By contrast, obtaining a reference to a filesystem object by opening it with the ''O_RDONLY'' flag requires that the caller have read permission on the object, even when the subsequent operation (e.g., ''fchdir(2)'', ''fstat(2)'') does not require read permission on the object.
  
If for example Samba receives an SMB request to open a file requesting ~SEC_FILE_READ_ATTRIBUTE~ access rights because the client wants to read the file's metadata from the handle, Samba will have to call ~open()~ with at least ~O_RDONLY~ access rights.
+
If for example Samba receives an SMB request to open a file requesting ''SEC_FILE_READ_ATTRIBUTE'' access rights because the client wants to read the file's metadata from the handle, Samba will have to call ''open()'' with at least ''O_RDONLY'' access rights.
  
 
== Usecases for O_PATH in Samba ==
 
== Usecases for O_PATH in Samba ==
The ~O_PATH~ flag is currently not used in Samba. By leveraging this Linux specific flags we can avoid permission mismatches as described above.
 
  
Additionally ~O_PATH~ allows basing all filesystem accesses done by the fileserver on handle based syscalls by opening all client pathnames with ~O_PATH~ and consistently using for example ~fstat()~ instead of ~stat()~ throughout the codebase.
+
The ''O_PATH'' flag is currently not used in Samba. By leveraging this Linux specific flags we can avoid permission mismatches as described above.
 +
 
 +
Additionally ''O_PATH'' allows basing all filesystem accesses done by the fileserver on handle based syscalls by opening all client pathnames with ''O_PATH'' and consistently using for example ''fstat()'' instead of ''stat()'' throughout the codebase.
  
 
Subsequent parts of this document will call such file-handles opened with O_PATH *path referencing file-handles* or *pathref*s for short.
 
Subsequent parts of this document will call such file-handles opened with O_PATH *path referencing file-handles* or *pathref*s for short.
  
 
== When to open with O_PATH ==
 
== When to open with O_PATH ==
In Samba the decision whether to call POSIX ~open()~ on a client pathname or whether to leave the low-level handle at -1 (what we call a stat-open) is based on the client requested SMB acccess mask.
+
In Samba the decision whether to call POSIX ''open()'' on a client pathname or whether to leave the low-level handle at -1 (what we call a stat-open) is based on the client requested SMB acccess mask.
  
The set of access rights that trigger an ~open()~ includes ~READ_CONTROL_ACCESS~. As a result, the open() will be done with at least ~O_RDONLY~. If the filesystem supports NT style ACLs natively (like GPFS or ZFS), the filesystem may grant the user requested right ~READ_CONTROL_ACCESS~, but it may not grant ~READ_DATA~ (~O_RDONLY~).
+
The set of access rights that trigger an ''open()'' includes ''READ_CONTROL_ACCESS''. As a result, the open() will be done with at least ''O_RDONLY''. If the filesystem supports NT style ACLs natively (like GPFS or ZFS), the filesystem may grant the user requested right ''READ_CONTROL_ACCESS'', but it may not grant ''READ_DATA'' (''O_RDONLY'').
  
 
Currently the full set of access rights that trigger opening a file is:
 
Currently the full set of access rights that trigger opening a file is:
Line 101: Line 102:
  
 
== Fallback on systems without O_PATH support ==
 
== Fallback on systems without O_PATH support ==
The code of higher level file-handle consumers must be kept simple and streamlined, avoiding special casing the handling of the file-handles opened with or without ~O_PATH~. To achieve this, a fallback that allows opening a file-handle with the same higher level semantics even if the system doesn't support ~O_PATH~ is needed.
+
The code of higher level file-handle consumers must be kept simple and streamlined, avoiding special casing the handling of the file-handles opened with or without ''O_PATH''. To achieve this, a fallback that allows opening a file-handle with the same higher level semantics even if the system doesn't support ''O_PATH'' is needed.
  
The way this is implemented on such systems is impersonating the root user for the ~open()~ syscall. In order to avoid privelege escalations security issues, we must carefully control the use these file-handles.
+
The way this is implemented on such systems is impersonating the root user for the ''open()'' syscall. In order to avoid privelege escalations security issues, we must carefully control the use these file-handles.
  
The low level filehandle is stored in a public struct ~struct file_handle~ that is part of the widely used ~struct files_struct~. Consumers used to simply access the fd directly by derefencing pointers to ~struct files_struct~.
+
The low level filehandle is stored in a public struct ''struct file_handle'' that is part of the widely used ''struct files_struct''. Consumers used to simply access the fd directly by derefencing pointers to ''struct files_struct''.
  
 
In order to guard access to such file-handles we do two things:
 
In order to guard access to such file-handles we do two things:
 
* tag the pathref file-handles and
 
* tag the pathref file-handles and
* control access to the file-handle by making the structure ~struct file_handle~ private, only allowing access with accessor functions that implement a security boundary.
+
* control access to the file-handle by making the structure ''struct file_handle'' private, only allowing access with accessor functions that implement a security boundary.
  
 
In order to avoid bypassing restrictive permissions on intermediate directories of a client path, the root user is only impersonated after changing directory to the parent directory of the client requested pathname.
 
In order to avoid bypassing restrictive permissions on intermediate directories of a client path, the root user is only impersonated after changing directory to the parent directory of the client requested pathname.
  
Two functions can then be used to fetch the low-level system file-handle from a ~struct files_struct~:
+
Two functions can then be used to fetch the low-level system file-handle from a ''struct files_struct'':
* ~fsp_get_io_fd(fsp)~: enforces fsp is NOT a pathref file-handle and
+
* ''fsp_get_io_fd(fsp)'': enforces fsp is NOT a pathref file-handle and
* ~fsp_get_pathref_fd(fsp)~: allows fsp to be either a pathref file-handle or a traditional POSIX file-handle opened with O_RDONLY or any other POSIX open flag.
+
* ''fsp_get_pathref_fd(fsp)'': allows fsp to be either a pathref file-handle or a traditional POSIX file-handle opened with O_RDONLY or any other POSIX open flag.
  
Note that the name ~fsp_get_pathref_fd()~ may sound confusing at first given that the fsp can be either a pathref fsp or a "normal/full" fsp, but as any full file-handle can be used for IO and as path reference, the name correctly reflects the intended usage of the caller.
+
Note that the name ''fsp_get_pathref_fd()'' may sound confusing at first given that the fsp can be either a pathref fsp or a "normal/full" fsp, but as any full file-handle can be used for IO and as path reference, the name correctly reflects the intended usage of the caller.
  
 
== When to use fsp_get_io_fd() or fsp_get_pathref_fd() ==
 
== When to use fsp_get_io_fd() or fsp_get_pathref_fd() ==
 +
The general guideline is:
 +
* if you do something like ''fstat(fd)'', use ''fsp_get_pathref_fd()'',
 +
* if you do something like ''*at(dirfd, ...)'', use ''fsp_get_pathref_fd()'',
 +
* if you want to print the fd for example in ''DEBUG'' messages, use ''fsp_get_pathref_fd()'',
 +
* if you want to call ''close(fd)'', use ''fsp_get_pathref_fd()'',
 +
* if you're doing a logical comparison of fd values, use ''fsp_get_pathref_fd()''.
  
The general guideline is:
+
In any other case use ''fsp_get_io_fd()''.
 
 
- if you do something like ~fstat(fd)~, use ~fsp_get_pathref_fd()~,
 
 
 
- if you do something like ~*at(dirfd, ...)~, use ~fsp_get_pathref_fd()~,
 
 
 
- if you want to print the fd for example in =DEBUG= messages, use ~fsp_get_pathref_fd()~,
 
 
 
- if you want to call ~close(fd)~, use ~fsp_get_pathref_fd()~,
 
 
 
- if you're doing a logical comparison of fd values, use ~fsp_get_pathref_fd()~.
 
 
 
In any other case use ~fsp_get_io_fd()~.
 

Latest revision as of 15:29, 14 January 2021

Summary

The effort to modernize Samba's VFS interface has reached a major milestone with the next release Samba 4.14.

Starting with version 4.14 Samba provides core infrastructure code that allows basing all access to the server's filesystem on file handles and not on paths. An example of this is using fstat() instead of stat(), or SMB_VFS_FSTAT() instead of SMB_VFS_STAT() in Samba parlance.

Historically Samba's fileserver code had to deal a lot with processing path based SMB requests. While the SMB protocol itself has been streamlined to be purely handle based starting with SMB2, large parts of infrastructure code remains in place that will "degrade" handle based SMB2 requests to path based filesystem access.

In order to fully leverage the handle based nature of the SMB2 protocol we came up with a straight forward way to convert this infrastructure code.

At the core, we introduced a helper function that opens a file handle that only serves as a path reference and hence can not be used for any sort of access to file data.

Samba's internal file handle structure is of type struct files_struct and all variable pointing to objects of such type are typically called fsp. Until very recently the only function that would open such a file handle and return an fsp was SMB_VFS_CREATE_FILE().

Internally SMB_VFS_CREATE_FILE() consisted of processing through Samba's VFS open function to open the low level file and then going through Samba's Windows NTFS emulation code.

The key point of the new helper function which is called openat_pathref_fsp() is that it skips the NTFS emulation logic. Additionally, the handle is restricted internally to be only usable as a path reference but not for any sort of IO. On Linux this is achieved by using the O_PATH open() flag, on systems without O_PATH support other mechanisms are used described in more detail below.

Path processing in Samba typically means processing client supplied paths by Samba's core path processing function filename_convert() which returs a pointer to an object of type struct smb_filename. Pointers to such objects are then passed around, often passing many layers of code.

By attaching an fsp file handle returned from openat_pathref_fsp() to all struct smb_filename objects returned from filename_convert(), the whole infrastructure code has immediate access to a file handle and so the large infrastructure codebase can be converted to use handle based VFS functions whenever VFS access is done in a piecemeal fashion.

Samba and O_PATH

Background

On Linux the O_PATH flag to open() can be used to open a filehandle on a file or directory with interesting properties: [fn:manpage]

  • the file-handle indicates a location in the filesystem tree,
  • no permission checks are done by the kernel on the filesystem object and
  • only operations that act purely at the file descriptor level are allowed.

The file itself is not opened, and other file operations (e.g., read(2), write(2), fchmod(2), fchown(2), fgetxattr(2), ioctl(2), mmap(2)) fail with the error EBADF.

The following subset of operations that is relevant to Samba is allowed:

  • close(2),
  • fchdir(2), if the file descriptor refers to a directory,
  • fstat(2),
  • fstatfs(2) and
  • passing the file descriptor as the dirfd argument of openat() and the other "*at()" system calls. This includes linkat(2) with AT_EMPTY_PATH (or via procfs using AT_SYMLINK_FOLLOW) even if the file is not a directory.

Opening a file or directory with the O_PATH flag requires no permissions on the object itself (but does require execute permission on the directories in the path prefix). By contrast, obtaining a reference to a filesystem object by opening it with the O_RDONLY flag requires that the caller have read permission on the object, even when the subsequent operation (e.g., fchdir(2), fstat(2)) does not require read permission on the object.

If for example Samba receives an SMB request to open a file requesting SEC_FILE_READ_ATTRIBUTE access rights because the client wants to read the file's metadata from the handle, Samba will have to call open() with at least O_RDONLY access rights.

Usecases for O_PATH in Samba

The O_PATH flag is currently not used in Samba. By leveraging this Linux specific flags we can avoid permission mismatches as described above.

Additionally O_PATH allows basing all filesystem accesses done by the fileserver on handle based syscalls by opening all client pathnames with O_PATH and consistently using for example fstat() instead of stat() throughout the codebase.

Subsequent parts of this document will call such file-handles opened with O_PATH *path referencing file-handles* or *pathref*s for short.

When to open with O_PATH

In Samba the decision whether to call POSIX open() on a client pathname or whether to leave the low-level handle at -1 (what we call a stat-open) is based on the client requested SMB acccess mask.

The set of access rights that trigger an open() includes READ_CONTROL_ACCESS. As a result, the open() will be done with at least O_RDONLY. If the filesystem supports NT style ACLs natively (like GPFS or ZFS), the filesystem may grant the user requested right READ_CONTROL_ACCESS, but it may not grant READ_DATA (O_RDONLY).

Currently the full set of access rights that trigger opening a file is:

  • FILE_READ_DATA
  • FILE_WRITE_DATA
  • FILE_APPEND_DATA
  • FILE_EXECUTE
  • WRITE_DAC_ACCESS
  • WRITE_OWNER_ACCESS
  • SEC_FLAG_SYSTEM_SECURITY
  • READ_CONTROL_ACCESS

In the future we can remove the following rights from the list on systems that support O_PATH:

  • WRITE_DAC_ACCESS
  • WRITE_OWNER_ACCESS
  • SEC_FLAG_SYSTEM_SECURITY
  • READ_CONTROL_ACCESS

Fallback on systems without O_PATH support

The code of higher level file-handle consumers must be kept simple and streamlined, avoiding special casing the handling of the file-handles opened with or without O_PATH. To achieve this, a fallback that allows opening a file-handle with the same higher level semantics even if the system doesn't support O_PATH is needed.

The way this is implemented on such systems is impersonating the root user for the open() syscall. In order to avoid privelege escalations security issues, we must carefully control the use these file-handles.

The low level filehandle is stored in a public struct struct file_handle that is part of the widely used struct files_struct. Consumers used to simply access the fd directly by derefencing pointers to struct files_struct.

In order to guard access to such file-handles we do two things:

  • tag the pathref file-handles and
  • control access to the file-handle by making the structure struct file_handle private, only allowing access with accessor functions that implement a security boundary.

In order to avoid bypassing restrictive permissions on intermediate directories of a client path, the root user is only impersonated after changing directory to the parent directory of the client requested pathname.

Two functions can then be used to fetch the low-level system file-handle from a struct files_struct:

  • fsp_get_io_fd(fsp): enforces fsp is NOT a pathref file-handle and
  • fsp_get_pathref_fd(fsp): allows fsp to be either a pathref file-handle or a traditional POSIX file-handle opened with O_RDONLY or any other POSIX open flag.

Note that the name fsp_get_pathref_fd() may sound confusing at first given that the fsp can be either a pathref fsp or a "normal/full" fsp, but as any full file-handle can be used for IO and as path reference, the name correctly reflects the intended usage of the caller.

When to use fsp_get_io_fd() or fsp_get_pathref_fd()

The general guideline is:

  • if you do something like fstat(fd), use fsp_get_pathref_fd(),
  • if you do something like *at(dirfd, ...), use fsp_get_pathref_fd(),
  • if you want to print the fd for example in DEBUG messages, use fsp_get_pathref_fd(),
  • if you want to call close(fd), use fsp_get_pathref_fd(),
  • if you're doing a logical comparison of fd values, use fsp_get_pathref_fd().

In any other case use fsp_get_io_fd().