Samba4/s3fs

From SambaWiki

What is s3fs

s3fs is the name that was given to the development effort to make possible the agreed default file server configuration for Samba 4.0 as an AD Domain controller.

It was agreed at SambaXP 2010 that Samba 4.0 would release with the smbd file server in use by default, so that users upgrading from Samba 3.x DC environments to the 4.0 AD DC would still have access to all the features of the smbd file server environment that they had come to expect and rely on.

Why is this called s3fs?

The primary additional feature that s3fs gives our users is access to the SMB3 protocol developed in smbd. Additionally, this name communicates that this is about using the smbd file server from the Samba 3.x release stream.

For a number of years, two file servers were developed in parallel in the Samba project. The Samba 3.x releases produced the smbd file server, while the Samba 4.x alpha series had the ntvfs file server. The ntvfs server never gained more than early SMB2 support. Therefore, s3fs simply indicates that, integrated into the samba process used by the AD DC is the file server refined in the Samba 3.x release series and supporting SMB3.

It was hoped that this detail and name would not need to be exposed to users by the time that Samba 4.x was released, while we tried to refine the development and implementation details of this process.

How it is implemented

Because the smbd file server has a distinct history from the ntvfs file server that the Samba4 development project has used so far, it did not naturally use the same security subsystems and other resources common the the rest of the AD server. Indeed, at a time before the Franky effort was started, it was not even in the same GIT tree, and certainly not the same build tree.

The merged build

The merged or top level build is the Waf build system. Running ./configure and make at the top level of the source tree will build all of Samba, including the previously Samba3 components. As all duplicate symbol names have been merged, renamed or otherwise dealt with, the merged build has a large number of shared libraries that the whole project builds on, particularly the IDL generated file built using PIDL.

Common structures

Key structures describing the security, authentication and authorization state were changed to be in common. In particular, key structures like struct auth_session_info and struct security_token now describe the authentication and authorization state across the whole project.

Common subsystems

As part of the process of building s3fs, a large number of subsystems were made common, including in particular GENSEC. smbd now uses gensec for all 'blob-based' server-side authentication, in all protocols, which has made it possible to both move to a proper implementation of SPNEGO based on GSSAPI, and allowed the activated plugins to be easily switched for AD DC operation.

Named pipe forwarding

DCE/RPC servers in Samba have in the past been implemented in-process, simply being additional library code that implements the required service, within the forked smbd child process. However, for Samba 4.0 as an AD DC, there are technical reasons why the DCE/RPC servers need to be implemented in a long-lived manner. To communicate between the smbd process that handles file sharing and the DCE/RPC server, all the SMB named pipe operations are converted into operations on a unix domain socket. For AD services, other parts of the samba binary then listen on these private unix domain socket connections for SMB named pipe requests.

This also has the advantage that the AD services from the Samba4 heritage do not need to directly link into the smbd file server or the associated DCE/RPC stack. This approach was originally developed for the Franky effort for this reason, and remains key to how DCE/RPC is handled in Samba 4.0.

Other DCE/RPC services, such as spoolss are not forwarded, or are forwarded to another part of the smbd binary, allowing printing to function on an AD DC.

smb.conf configuration

The automatically generated fileserver.conf has lines such as:

rpc_server:samr = external

to indicate that these pipes are handled outside the smbd binary.

Plugins

With the merged build, it then becomes possible to load or link in plugins for key subsystems, to change the behaviour of the smbd file server, as required for consistency with the rest of the AD server.

auth_samba4

perhaps the most key plugin in auth_samba4. This plugin provides 2 very important hooks.

prepare_gensec

prepare_gensec returns a struct gensec_security that is pre-initialised to use the GENSEC modules that the rest of the AD server uses, including the authentication contexts etc required to authenticate against the AD directory in LDB. When the auth_samba4 module is not in use, the fallback is to the default authentication path, also expressed as a series of GENSEC modules.

make_auth4_context

make_auth4_context returns a struct auth4_context that similarly operates like the rest of the AD server, for SessionSetupAndX calls that do not use SPNEGO or NTLMSSP.

pdb_samba4

For operation of smbpasswd, pdbedit and net sam tools, and to assist in the migration of Samba 3.x DC environments to Samba 4.0 AD, a passdb module was written based on pdb_ads from Franky. This module differs in that it directly calls ldb and the associated modules, allowing offline operation when the samba server process is not running. This is particularly critical for the samba-tool domain samba3upgrade tool.

vfs_samba4

This module makes the ldb calls required to implement domain DFS referrals on the AD DC, and is loaded for operation on IPC$.

IDMAP

The idmap code in smbd has been modified to accept an ID mapping type of IDMAP_BOTH, representing both a uid and a gid. This will then allow a group (such as Domain Admins) to own a file as a uid, but also be a gid when expressed as group membership.

Winbind

Currently in s3fs, the winbindd implementation in use is that in the samba binary, from the Samba4 heritage. It shares the same protocol as the Samba3 winbind in the winbindd binary, but does not implement the full protocol. It also implements a private IRPC based protocol for communication with other parts of the samba binary.

Starting s3fs

s3fs is started by the code in file_server/file_server.c, by writing a private fileserver.conf that includes a preamble of smb.conf options required for operation in this mode, and then it starts the smbd binary by fork() and exec().

At provision time, it is the default. To use the old ntvfs server, simply add the option:

--use-ntvfs

Or on later versions:

--with-ntvfs-fileserver

For instance run:

 ./provision --domain=DEMOS3FS --realm=demos3fs.samba.corp --adminpass=P@ssw0rd

This creates an smb.conf with the equivalent of:

server services = -smb +s3fs
dcerpc endpoint servers = -winreg -srvsvc

You should add these options manually if you wish to move to s3fs on an existing production system.

Do note that NT ACLs set by s3fs will not be read by the ntvfs file server, if you switch back. This is because while both use the security.NTACL extended attribute, the s3fs module writes format version 3, not the original version 1 understood by the NTVFS server.

Testing

s3fs is tested in the plugin_s4_dc environment in make test. More tests need to be added

Using it

s3fs is the default. To use the previous default, the ntvfs server use --use-ntvfs at provision time or add these lines from the smb.conf

server services = +smb -s3fs
dcerpc endpoint servers = +winreg +srvsvc