UNIX Extensions

From SambaWiki

Also see http://samba.org/samba/CIFS_POSIX_extensions.html.

Introduction

Note that the CIFS dialect is being deprecated, and that POSIX extensions for the current, and much more secure, version of the protocol family (SMB3.11 dialect) are defined here: SMB3 Unix Specification. POSIX (Unix and Linux) compatibility for the current SMB3 version of the protocol, and current protocol extensions, are described here: SMB3 POSIX Extensions

The Unix Extensions to the CIFS Protocol have been done in stages. An initial set which included various new infolevels to TRANSACT2 commands in the range from 0x200 to 0x2FF (inclusive), was available when:

   CAP_UNIX (0x00800000)

was included in the SMB negotiate protocol response.

Additional POSIX extensions have been added based on negotiating individual capabilities on the tree connection (via a Unix QueryFSInfo and SetFSInfo level). Following is a list of the capabilties which may be negotiated:

Negotiating per-share (tree connection) Capabilities

Capability Value
CIFS_UNIX_FCNTL_LOCKS_CAP 0x001
CIFS_UNIX_POSIX_ACLS_CAP 0x002
CIFS_UNIX_XATTR_CAP 0x004
CIFS_UNIX_EXATTR_CAP 0x008
CIFS_UNIX_POSIX_PATHNAMES_CAP 0x010 All characters except '/' should be supported in pathnames
CIFS_UNIX_POSIX_PATH_OPERATIONS_CAP 0x020
CIFS_UNIX_LARGE_READ_CAP 0x040 We can cope with 24 bit reads in readX.
CIFS_UNIX_LARGE_WRITE_CAP 0x080 We can cope with 24 bit writes in writeX.
CIFS_UNIX_TRANSPORT_ENCRYPTION_CAP 0x100 We can do SPNEGO negotiations for encryption.
CIFS_UNIX_TRANSPORT_ENCRYPTION_MANDATORY_CAP 0x200 We *must* SPNEGO negotiations for encryption.
CIFS_UNIX_PROXY_CAP 0x400 Proxy capability, supports 0xACE ntioctl and QFS PROXY call

New File Info (and Path Info) levels

Capability Value Description
QUERY_FILE_UNIX_BASIC 0x200 Part of the initial Unix Extensions
QUERY_FILE_UNIX_LINK 0x201 Part of the initial Unix Extensions
QUERY_POSIX_ACL 0x204 Requires CIFS_UNIX_POSIX_ACL_CAP, MUST be supported if set
QUERY_XATTR 0x205 Requires CIFS_UNIX_XATTR_CAP, MUST be supported if set
QUERY_ATTR_FLAGS 0x206 Requires CIFS_UNIX_EXTATTR_CAP, MUST be supported if set
QUERY_POSIX_PERMISSION 0x207
QUERY_POSIX_LOCK 0x208 Requires CIFS_UNIX_FCNTL_CAP, MUST be supported if set
SMB_POSIX_PATH_OPEN 0x209 Requires CIFS_UNIX_POSIX_PATH_OPERATIONS_CAP, MUST be supported if set
SMB_POSIX_PATH_UNLINK 0x20a Requires CIFS_UNIX_POSIX_PATH_OPERATIONS_CAP, SHOULD be supported if set
SMB_QUERY_FILE_UNIX_INFO2 0x20b Requires CIFS_UNIX_EXTATTR_CAP, SHOULD be supported if set


Posix Open

The server specifies it can serve these by returning CIFS_UNIX_POSIX_PATH_OPERATIONS_CAP in the reply to a trans2 qfsinfo (TRANSACT2_QFSINFO 0x03) info level SMB_QUERY_CIFS_UNIX_INFO (0x200) call. All values are little endian.

For open, call TRANSACT2_SETPATHINFO (command 0x06) info level :

SMB_POSIX_PATH_OPEN 0x209

The request data block should be 18 bytes consisting of the following :

Size Offset Value
4 bytes 0 Flags field (same as smb_ntcreate_flags in SMBNTCreateX to request oplocks)
4 bytes 4 POSIX open flags (see below). (for mkdir specify O_CREAT O_DIRECTORY)
8 bytes 8 POSIX Permissions (see below)
2 bytes 16 Reply info level requested (see below)

Encodings are as follows :

POSIX open flags Value
SMB_O_RDONLY 0x1
SMB_O_WRONLY 0x2
SMB_O_RDWR 0x4
SMB_O_CREAT 0x10
SMB_O_EXCL 0x20
SMB_O_TRUNC 0x40
SMB_O_APPEND 0x80
SMB_O_SYNC 0x100
SMB_O_DIRECTORY 0x200
SMB_O_NOFOLLOW 0x400
SMB_O_DIRECT 0x800
POSIX permissions Value Description
SMB_PERM_SUID 0004000 set UID bit
SMB_PERM_SGID 0002000 set-group-ID bit
SMB_PERM_SVTX 0001000 sticky bit
SMB_PERM_RUSR 00400 owner has read permission
SMB_PERM_WUSR 00200 owner has write permission
SMB_PERM_XUSR 00100 owner has execute permission
SMB_PERM_RGRP 00040 group has read permission
SMB_PERM_WGRP 00020 group has write permission
SMB_PERM_XGRP 00010 group has execute permission
SMB_PERM_ROTH 00004 others have read permission
SMB_PERM_WOTH 00002 others have write permission
SMB_PERM_XOTH 00001 others have execute permission

The response data block varies in length depending on the level requested :

Size Offset Value
2 bytes 0 Flags field (same flags in as oplock response field in SMBNTCreateX, although bigger field)
2 bytes 2 Network file handle (fid). Zero is returned in this field for mkdir case.
4 bytes 4 CreateAction (same as in NTCreateX response, might not be meaningful for directories)
2 bytes 8 Reply Information level returned (see below)
2 bytes 10 Pad (Must be zero)
(sizeof reply information) 12 when Reply information level is not SMB_NO_INFO_LEVEL_RETURNED (ie not 0xFFFF)

TBD: How do we return the Create Action (File Created vs. File Opened)?

Posix Unlink

Posix and Windows semantics for unlink of open files are different. POSIX allows deleting open files (which has the effect of removing them from the directory listing, preventing them from being opened again, but allowing existing users who have the file open to continue to read and write from the existing handle(s) until the handle(s) are closed when the inode or equivalent is deleted from the server).

If the DOS readonly bit is set on the file, the unlink must fail.

UNIX_INFO2

The UNIX_INFO2 is an extension to the UNIX_BASIC info level. This info level can be used in FindFirst/FindNext, QueryPathInfo, QueryFileInfo and PosixOpen (but is not restricted to those calls).

Info Level Value
SMB_QUERY_FILE_UNIX_INFO2 0x20B
SMB_SET_FILE_UNIX_INFO2 0x20B
SMB_FIND_FILE_UNIX_INFO2 0x20B

The response block for the UNIX_INFO2 level is below. This is deliberately defined to be the same as UNIX_BASIC except for the last 3 fields.

Size Offset Type Name Description
8 0 LARGE_INTEGER EndOfFile File size
8 8 LARGE_INTEGER Blocks Number of blocks used on disk
8 16 LARGE_INTEGER ChangeTime Attribute change time
8 24 LARGE_INTEGER LastAccessTime Last access time
8 32 LARGE_INTEGER LastModificationTime Last modification time
8 40 LARGE_INTEGER Uid Numeric user id for the owner
8 48 LARGE_INTEGER Gid Numeric group id of owner
4 56 ULONG Type Enumeration specifying the file type
8 60 LARGE_INTEGER DevMajor Major device number if type is device
8 68 LARGE_INTEGER DevMinor Minor device number if type is device
8 76 LARGE_INTEGER UniqueId This is a server-assigned unique id
8 84 LARGE_INTEGER Permissions Standard UNIX permissions
8 92 LARGE_INTEGER NumLinks Number of hard links
8 100 LARGE_INTEGER CreationTime Create/birth time
4 108 ULONG FileFlags File flags enumeration
4 112 ULONG FileFlagsMask Mask of valid flags

If the client is doing a set with the UNIX_INFO2 level and it does not want to alter the FileFlags, it should provide a FileFlagsMask of 0.

The defined set of file flags is

File Flag Value Interpretation
EXT_SECURE_DELETE 0x00000001 File should be erased such that the data is not recoverable
EXT_ENABLE_UNDELETE 0x00000002 File should opt-in to a server-specific deletion recovery scheme
EXT_SYNCHRONOUS 0x00000004 I/O to this file should be performed synchronously
EXT_IMMUTABLE 0x00000008 NO changes can be made to this file
EXT_OPEN_APPEND_ONLY 0x00000010 Only appends can be made to this file
EXT_DO_NOT_BACKUP 0x00000020 Backup programs should ignore this file
EXT_NO_UPDATE_ATIME 0x00000040 The server is not required to update the last access time on this file
EXT_HIDDEN 0x00000080 User interface programs may ignore this file

Unlike the UNIX_BASIC infolevel, the UNIX_INFO2 infolevel response for FindFirst/FindNext includes a 4 byte name length field immediately before the file name field.

SMB_FIND_FILE_UNIX_INFO2 response:

Size Offset Type Name Description
4 0 ULONG NextEntryOffset
4 4 ULONG FileIndex
116 8 UNIX_INFO2 UnixInfo2 UNIX_INFO2 structure as defined above
4 124 ULONG FileNameLength length of filename in bytes (not including any terminating NULL)
... 128 STRING FileName file name (does not include any terminating NULL)

For FindFirst/FindNext the new UnixInfo2 structure begins (as some of the other FindFirst/FindNext levels do) with

4 bytes NextEntryOffset 4 bytes ResumeKey

QUERY_FILE_UNIX_BASIC

Described in the SNIA CIFS Technical Reference.

QUERY_FILE_UNIX_LINK

Described in the SNIA CIFS Technical Reference.

QUERY_POSIX_ACL

<Under Construction>

SMB_QUERY_ATTR_FLAGS

<Under Construction>

For setting file attribute flags (see man page for lsattr/chflags and equivalent)

Proposed format:

       __le64  mask; /* list of all possible attribute bits */
       __le64  mode; /* list of actual attribute bits on this inode */
FILE_CHATTR_INFO /* ext attributes (chattr, chflags) level 0x206 */

QUERY_XATTR

<Under Construction>

xattrs (extended attributes) are of the form

  namespace.key = value


Current xattrs in the "user" (also known as "OS2") namespace can readily map to SMB/CIFS EAs by simply stripping off the "user." in the namespace (prefix) sending only the key and value.


Some operating systems define additional classes of extended attribute (name/value pairs) which may be associated with an inode, and are available to be set by administrative users. Such classes of extended attributes include the "trusted" and "security" namespaces.

Proposed format:

       /* Do we need another field for flags? */
       __u32 xattr_name_len;
       __u32 xattr_value_len;
       char  xattr_name[0];
       /* followed by xattr_value[xattr_value_len], no pad */

FILE_XATTR_INFO /* extended attribute, info level 0x205 */

Sending attributes in the other namespace categories requires this new trans2 info level.

Note that the server may associate different default ACL permissions on xattrs in different namespaces on the same inode.

New Query/Set FS Info levels: Operations on shares/exports

Info Level Name Value Description
SMB_QUERY_CIFS_UNIX_INFO 0x200 (Part of the original Unix Extensions)
SMB_QUERY_POSIX_FS_INFO 0x201
SMB_QUERY_POSIX_WHO_AM_I 0x202 (see below)
SMB_REQUEST_TRANSPORT_ENCRYPTION 0x203 Call to set up an encryption context.
SMB_QUERY_FS_PROXY 0x204 New NTIOCTL available (0xACE) for WAN friendly SMB (see below)

QUERY_CIFS_UNIX_INFO

size Offset Field Description
2 0 Major Version Number
2 2 Minor Version Number
8 4 Capability Flags

Many current servers return Major Version 1, Minor Version 0

SET_CIFS_UNIX_INFO

<Under Construction>

SMB_QUERY_POSIX_FS_INFO

The statfs command on many operating systems distinguishes between the number of bytes available on the volume to regular users and the number of bytes available on the volume for administrative users. In addition, the total number of inodes (nodes, vnodes) on the volume, is often reported as well. This new QFS Info level returns sufficient information to fill in the most important fields in the common statfs call.

size Offset Field Description
4 0 Optimal Transfer Size (bsize on some operating systems)
4 4 Blocksize
8 8 Total Blocks
8 16 Blocks Available
8 24 User Blocks Available
8 32 Total File Nodes
8 40 Free File Nodes
8 48 File System Identifier (fsid)

Note that the other fields in the common form of the local stat call can come from existing QFS Info levels

SMB WHOAMI

The SMBWhoami extension is intended to be a lightweight method for a Unix client to be able to display sensible file ownership information.

SMBWhoami is performed by requesting a TRANS2_QFSINFO with an info level of SMB_QUERY_POSIX_WHOAMI. There are no parameters passed. The vuid (and optionally the tid) field is implicitly used.

Info Level Name Value
SMB_QUERY_POSIX_WHOAMI 0x202

The first two fields of the SMBWhoami response are a set of flags that further describe how the server has mapped the connected user. The mask is returned so that the client can distinguish which flag bits are meaningful.

Mapping Flag Value Interpretation
SMB_WHOAMI_GUEST 0x1 Logged in as (or squashed to) guest
Size Offset Value
4 bytes 0 Mapping flags
4 bytes 4 Mask of valid mapping flags
8 bytes 8 Primary user ID
8 bytes 16 Primary group ID
4 bytes 24 number of supplementary GIDs
4 bytes 28 number of SIDs
4 bytes 32 SID list byte count
4 bytes 36 Reserved (should be zero)
variable ... list of 8 byte group IDs (may be empty)
variable ... List of DOM_SID structures (may be empty)

Note that the list of group IDs and DOM_SIDs are both optional. A server may choose not to return these (eg. if the information is expensive to gather). If these are not returned the corresponding count fields must be zero.

SMB FS Proxy

The SMB Proxy extension provides a set of extended read and write operations that make use of read-ahead, caching and compression to improve SMB performance over WAN environments.

Returns structure FILE_SYSTEM_UNIX_INFO to describe proxy version and capabilities.

The proxy transport is NTIOCTL with function code 0xACE (shifted left twice).

Function codes 0-2047 << 2 are reserved for Microsoft; 
  codes 2048-4095 << 2 are reserved for OEMs and IHVs.
  See: http://msdn2.microsoft.com/en-us/library/aa914767.aspx "CTL_CODE"

The ioctl payload consts of a little endian GUID, a 32 bit operation number and then some little endian NDR from generated IDL supporting the proxy operations.

DFS Side Effects

The use of reserved path characters such as backslash, colon, question mark and asterisk in DFS referrals can create interoperability problems. Many common clients and servers do not permit such characters in file or directory names. In particular many Windows servers do not support either '\' or '/' in path components.

Although the CIFS_UNIX_POSIX_PATHNAMES_CAP implied the ability to recognize the backslash ('\') as a valid character in a directory or file name (rather than treating backslash as a path component separator) it was not required, and some common servers also can not handle the backslash within directory names in the response processing for DFS requests (in particular for TRANS2_GET_DFS_REFERRAL, trans2 command 0x10). DFS referrals requests and responses include a pathname which may include multiple levels of subdirectories. When CIFS_UNIX_POSIX_PATHNAMES_CAP is negotiated the server MAY report DFS paths which point to certain target storage servers (those which are known to support CIFS_UNIX_POSIX_PATHNANMES_CAP) as:

   \<server>\<share>/directory/subdirectory/more-subdirectories

If the target storage server type is not known the server MAY canonicalize paths (replacing / with \) and report DFS paths as:

   \<server>\<share>\directory\subdirectory\more-subdirectories

since / is not a valid path separator on some target servers. The client can detect that the server has canonicalized paths because the character that immediately follows the share is a '\' rather than a '/' character. Note that share names MUST not contain either the '\' or '/' character.

Paths which contain components with embedded backslash are expected to be rare in practice.

Samba server version information

The data returned by the trans2 SMB_FS_OBJECTID_INFORMATION request contains 48 bytes of "extended information". Samba version 3.2 or later will return a samba_extended_info_version structure in this field.

/* Used in SMB_FS_OBJECTID_INFORMATION requests.  Must be exactly 48 bytes. */
#define SAMBA_EXTENDED_INFO_MAGIC 0x536d4261 /* "SmBa" */
#define SAMBA_EXTENDED_INFO_VERSION_STRING_LENGTH 28

struct smb_extended_info {
	uint32 samba_magic;		/* Always SAMBA_EXTRA_INFO_MAGIC */
	uint32 samba_version;		/* Major/Minor/Release/Revision */
	uint32 samba_subversion;	/* Prerelease/RC/Vendor patch */
	NTTIME samba_gitcommitdate;
	char   samba_version_string[SAMBA_EXTENDED_INFO_VERSION_STRING_LENGTH];
};

This extension was first proposed in this samba-technical thread. Like most (all?) SMB structures it is marshalled without any "holes" for alignment.

Storing symlinks on Windows servers

Minshall+French symlinks

Steve French and Conrad Minshall defined a file format for storing Unix symlinks on SMB volumes. This shall forever be known as the Minshall+French format. See smbfs_create_windows_symlink_data and smbfs_smb_windows_read_symlink in Apple's kernel/smbfs/smbfs_smb.c. (There are good comments in that file as well.)

The current macOS smbfs implementation uses Minshall+French symlinks when connected to SMB1 servers that don't support UNIX Extensions and SMB2+ servers that do not support reparse symlinks. Because Samba does not yet support reparse symlinks, the only way to work with real UNIX symlinks with macOS clients is to limit clients to SMB1. In Apple's smbfs, the decision about what kind of symlink to use is made in smbfs_create and smbfs_vnop_readlink in kernel/smbfs/smbfs_vnops.c.

The Minshall+French format is a sequence of newline separated fields:

XSym\n
char len[]\n
char md5sum[32]\n
char target[len]\n
  • XSym: the literal ASCII characters 'X', 'S', 'y', 'm'
  • len: the length of the symlink target name as an ASCII string, with leading 0's
  • md5sum: The MD5 hash of the link target name
  • target: the link target path

In addition, the target is padded out with ASCII space characters to a fixed length (1024 bytes). The actual symlink files on disk are a fixed length of 1067 bytes, which allows the client to easily identify them without having to open every file to see if it's a symlink.

Extended Attribute symlinks

http://marc.info/?l=samba-technical&m=120229726332475&w=2

This proposal to store symlink information in extended attributes has not been implemented in any known SMB server.

SMB transport encryption

CIFS transport encryption is only available in Samba's smbclient utility ("--encrypt" parameter) when mounted to Samba 3.2 or later.

The SMB3 protocol does include a transport encryption feature which is implemented in Windows and Samba server. Implementation is in progress for the kernel file system (cifs.ko) for this feature.