SoC/Ideas: Difference between revisions

From SambaWiki
(Remove some outdated proposals)
(Change a title to be one level higher)
Line 91: Line 91:
We still need a few more Samba4-specific administration utilities to make Samba4 useful in real life. A Summer of Code student would be expected to do a number of these (identifying new needs from deployments), but here are some suggestions:
We still need a few more Samba4-specific administration utilities to make Samba4 useful in real life. A Summer of Code student would be expected to do a number of these (identifying new needs from deployments), but here are some suggestions:


====Setup / provision GUI for Samba (and OpenChange)====
===Setup / provision GUI for Samba (and OpenChange)===


Setting up Samba and OpenChange currently requires running quite a lot of command line utilities. It should be easy for even a junior system administrator to set up Samba as a Domain Controller, RODC and general server.
Setting up Samba and OpenChange currently requires running quite a lot of command line utilities. It should be easy for even a junior system administrator to set up Samba as a Domain Controller, RODC and general server.

Revision as of 16:10, 24 March 2011

Google Summer of Code: Suggested Project ideas

The following are the Samba project ideas for Summer of Code. Of course you are free to come up with ideas not listed here. Please discuss the your planned project by either joining us on irc://irc.freenode.net/#samba-technical or by sending email to samba-technical@samba.org

Most of our projects will require C programming skills, but the Samba4 section has a couple of Python projects.

Samba 3

Add remote (RPC) support for Samba configuration

Since almost two years now, Samba has a registry based configuration backend: Configuration data is stored inside the registry key HKEY_LOCAL_MACHINE\Software\Samba\smbconf. Access to this configuration is available through a module that makes use of the "reg_api" module which is the backend code for direct local access to the registry database. The reg_api interface is similar to the WINREG rpc interface. This project would at first develop a common API for registry access local through reg_api and remote through the winreg rpc client code. This new module could be abstraced from the code of the utility "net rpc registry". With this new module, the registry configuration code could be enhanced to allow for remote configuration.

Here are some more details:

The basic idea behind the project is that it should be possible to use the same interface for local (direct db) and remote (rpc) access. One can see the difference between that direct local and the remote access very well when one compares the code of the "net registry" and the "net rpc registry" commands: this is utils/net_registry.c vs. utils/net_rpc_registry.c . The logic is very similar while the actual calls into registry are very different.

The rpc client implementation for WINREG are the functions of the form rpccli_winreg_foo(). So the idea is to also use the rpccli_winreg_foo() functions to access the local registry db instead of direclty using the corresponding reg_foo() functions from reg_api.c. How can this be achieved? Each access to a rpc service must use an rpc_client structure that is created by an rpc bind call (which connects to a remote server, authenticates and such). Now the trick is that there is already a sort of fake local rpc bind call in samba that makes the following rpccli_x_y calls not contact a remote server over the network but call out direclty to the local server implementation. This is the function rpc_pipe_open_internal() in rpc_server/srv_pipe_hnd.c .

So the idea is to create a function that uses this local bind for the WINREG service. And then maybe as a first exercise change "net registry" to use this bind and rpccli_winreg_ calls instead of reg_api calls. This will effectively make "net registry" a "--local" mode of the "net rpc registry" command.

As a second stage (returning to the remote configuration topic of the project description), note that the registry based configuration uses a subsystem called libsmbconf stored in lib/smbconf/ , which has several backends, one of which is registry. The registry backend implementation could then be changed to use such a bind and rpccli_winreg_* calls. This would then after some extension make it possible to use libsmbconf and hence "net conf" to configure a remote samba server.

In the sequel (probably not any more part of this gsoc project), more subsystems of Samba that use the registry could be converted to use the local-bind + rpccli_winreg scheme, until in the end, the reg_api code will only be used in the WINREG rpc server implementation. This will make it much more easy do change the registry backend code in the future.

  • Difficulty: Medium
  • Language(s): C
  • Possible Mentors: Michael Adam


Make SWAT for Samba3 pretty

SWAT is the Samba Web Administration Tool, written in 1998. It still looks very 1998 as well. SWAT should get a complete redesign to match what people expect from a web application these days. The code running the web front-end will certainly need to be adapted as well. It might be viable to completely reimplement SWAT as a standalone web-app, if the current code cannot be adapted to support all features needed with a reasonable effort.

  • Difficulty: Medium
  • Language(s): JavaScript, C / possibly other language.
  • Possible mentors: Kai Blin

Samba 4

Admin Utilities

We still need a few more Samba4-specific administration utilities to make Samba4 useful in real life. A Summer of Code student would be expected to do a number of these (identifying new needs from deployments), but here are some suggestions:

Setup / provision GUI for Samba (and OpenChange)

Setting up Samba and OpenChange currently requires running quite a lot of command line utilities. It should be easy for even a junior system administrator to set up Samba as a Domain Controller, RODC and general server.

The windows equivalent tool is dcpromo.exe, and while you may not want/need to copy it directly, it does provide a reasonable basis for establishing what this tool should be able to provide (in terms of outcomes).

Some initial work (using PyQt4) is available at http://gitorious.org/samba-openchange-gui-tools/samba-openchange-gui-setup but working with that is not essential to this task.

  • Difficulty: Easy
  • Language(s): Python
  • Possible mentors: Brad Hards, probably others.

Windows Search Protocol WSP client library and torture tests

The Windows Search Protocol WSP is used to implement remote full filesystem indexing (indexed search) between windows machines. We would like to support this functionality in Samba, interfacing with existing indexing tools on Unix systems (such as beagle).

This is a new protocol based on SMB named pipes \pipe\ci_skads or \pipe\MSFTEWDS. See http://msdn2.microsoft.com/en-us/library/cc216195.aspx.

The student should write a (un)marshalling library to push and pull PDUs and an async client library on top of the samba4 raw smb client library.

The student should write sub tests for smbtorture which should demostrate how the protocol works against a windows server.

The student doesn't have to implement the samba4 server code.

  • Difficulty: Hard
  • Language(s): C, (Python?)
  • Possible Mentors: Tridge

Browsing support in Samba 4

Samba 4 still needs support for mailslots in general and in particular for the BROWSE mailslot. Should come with tests. Documentation of the BROWSER protocol is available here: http://msdn.microsoft.com/en-us/library/cc201609(PROT.10).aspx http://ubiqx.org/cifs/Browsing.html

  • Difficulty: Hard
  • Language(s): C
  • Possible mentors: JelmerVernooij, Stefan Metzmacher

Implement server side GPO in Samba4

Currently Samba4 supports GPOs, but setting them requires calling different python scripts manually to make sure the changes are propagated from the DC to the clients. The student should implement support in Samba that would allow using existing tools like gpmc.msc to set GPOs that are propagated to the clients. More details can be found in Matthieu's samba-technical email.

  • Difficulty: Medium
  • Language(s): C
  • Possible mentors: Matthieu Patou, Matthias Dieter Wallnöfer

Implement login / logout related counter update

For the moment the attributes related to login and logout are not updated by Samba4. The goal of this project is to understand in which case windows update the counters (ie. most probably during interactive logon but also maybe with some netlogon calls ?) and to implement counter and timestamp update is Samba code so that this information can be available. This project of course includes the development of unit tests.

  • Difficulty: Easy
  • Language(s): C
  • Possible mentors: Andrew Bartlett

Linux Kernel CIFS/SMB2 client improvements

Interested students should contact Steve French or Jeff Layton and discuss possible improvements to the Linux Kernel CIFS VFS client. Here are some ideas to get you started:

Improved async/vectored i/o support (improves performance)

  • add ability for cifs to issue calls in parallel and handle read/write responses asynchronously. Here's how JLayton envisions this, but there may be other ways to do it, there ar some other concerns to consider as well:
    • generalize callback mechanism for cifsd when it receives a response. Instead of just waking up a process, it could do more to handle a SMB response.
    • cifs_writepages sends write requests serially now. It should instead issue the requests rapid-fire. Don't wait for the response from the previous request before sending another.
    • when sending these requests, set it up so that cifsd will handle the response itself. As long as it doesn't block, it should be ok to have cifsd mark the pages clean, etc. Alternately, the write response handling could be offloaded to a workqueue or something.
    • this will allow cifs_writepages to return w/o waiting for a response when we're writing in the background (WB_SYNC_NONE case).
    • consider how to handle reconnection events, canceling requests, lack of server response, etc.
    • readpages could probably also use similar handling.
    • You might also consider merging write requests, etc. (similar to how NFS handles this)
    • should we move to an unstable pages model (like NFS)?
  • Language: C
  • Difficulty: Hard
  • Possible Mentors: Jeff Layton or Steve French


SMB2 protocol improvements

  • The SMB2 protocol (followon to cifs) adds many useful performance enhancements and new features. The Linux kernel implementation is still experimental and is missing key features including SMB2.1 **dialect support (items such as lease keys) and lacks a useful credit request algorithm (which Samba server only recently added for the server side). Various performance optimizations (including **support for very large reads and writes and dispatch of more requests in parallel) are also possible.
  • Language: C
  • Difficulty: Varies, Medium to Hard
  • Possible Mentors: Steve French

Support for SELinux

  • Mac Security Label support is important for virtualization and useful for improved security some workloads. Support for setting/getting these labels over the wire was investigated in the NFS version 4 workgroup. Adding support to the CIFS Unix Extensions (Linux kernel client and Samba server) should be possible, especially if this is just a new class of extended attribute. The goal would be to support this feature of SELinux to allow KVM and other applications to take advantage of security labels. Some of the background requirements are loosely related to the (nfs equivalent of) what is mentioned in: http://tools.ietf.org/html/draft-quigley-nfsv4-sec-label-01
  • Language: C
  • Difficulty: Hard
  • Possible Mentors: Steve French

Create GUI or command-line tools for displaying /proc/fs/cifs statistics and and mount/session status

  • might also involve some cleanup of the in-kernel stats / status output
  • Language: some C (for kernel code), something else for GUI?
  • Difficulty: Easy
  • Possible Mentors: Steve French

Create a common uid mapping mechanism for Linux nfs and cifs vfs clients

  • or maybe just figure out a way to hook cifs up to rpc.idmapd
  • add a way for the client to remap the uids returned by the server to uids which would be valid on the client (or to a default if such uid does not exist).
  • This is helpful especially when the server supports the CIFS Unix Extensions and has different uids and gids mapping than the client
  • Difficulty: Hard
  • Possible Mentors: Jeff Layton or Steve French

VFS change notification support

  • add VFS support for calling into the filesystem when setting up notifications
  • add code to cifs/smb2 to set up and deal with notifications from the server in response to inotify/dnotify calls
  • Difficulty: Hard
  • Possible Mentors: Jeff Layton or Steve French

Support for retrieving snapshots, encrypted files, or compressed files from Windows

  • Difficulty: Medium
  • Possible Mentors: Steve French

cifs->Samba automated test facility

  • Do build verification similar to what we can now do with the Samba server and tools in the Samba build farm. Mounts from the Linux SMB2 and CIFS kernel clients could be tested with posix file i/o tests which might include modified versions of the "connectathon" and xfstest test suites and others. The goal is to quickly identify problems with newly integrated patches.
  • Difficulty: Hard
  • Possible Mentors: Jeff Layton or Steve French

build infrastructure for storing NTLM creds in kernel keyring

  • so that the kernel can establish sessions on the fly without needing to prompt for passwords
    • primarily a companion project to multisession mounts, so that they don't require kerberos. It may be useful for other things too -- "regular" mounts possibly (in lieu of credential files), may even be something that could be used by smbclient.
    • CIFS already uses the keyctl API to do upcalls for SPNEGO and DNS requests for DFS. This would use the same API, but a little differently -- the idea would be for users to "stash" credentials in the keyring. The kernel would then be able to scrape those credentials out of the keyring whenever it needed to establish a session.
    • a PAM module might also be nice for this (to allow stashing of creds on login)
    • candidates should read the keyctl(2) and related manpages, and the keys.txt and keys-request-key.txt files in the Documentation/ directory in the Linux kernel sources for more info
  • Difficulty: Medium
  • Possible Mentors: Jeff Layton

Other Random Ideas

  • Ideas aren't limited to these, feel free to propose something else:
    • Create a GUI for creating and managing Linux cifs mounts, and more easily configuring the many complex cifs mount options, statistics (/proc/fs/cifs)
    • Support for alternate transport protocols (other than TCP sockets). Adding support for SCTP to cifs/smb2 kernel clients and Samba server or perhaps more interesting add support for Linux's "virtio" transport to the cifs/smb2 kernel clients and Samba server (to allow optimized mounts and zero-copy transfer of data from virtualized guests to hosts on the same box)
    • Support for features (such as directory delegations) which NFS version 4.1 has but which current CIFS even with the most current CIFS->Samba protocol extensions (CIFS Unix Extensions) do not have -- will probably need server support too.
    • Add additional library support or modify Samba client libraries so they can use existing kernel cifs functions (such as sending SMBs on negotiated sessions when the kernel client already has a session to the server). With the addition of library to access cifs's pipe (in kernel), Samba client libraries or other dce/rpc code could use cifs kernel sessions for management of and over cifs mounts.
    • Add libraries and utilities to manage acls (cifs kernel client has an extended attribute for setting/getting "raw" cifs acls but userspace posix acl tools obviously can't be used to manage cifs specific acl features).
  • Difficulty: Varies
  • Language(s): C
  • Possible mentors: Steve French