Difference between revisions of "SoC/Ideas"

(SMB2 protocol improvements)
Line 94: Line 94:
 
Interested students should contact Steve French or Jeff Layton and discuss possible improvements to the Linux Kernel CIFS VFS client. Here are some ideas to get you started:
 
Interested students should contact Steve French or Jeff Layton and discuss possible improvements to the Linux Kernel CIFS VFS client. Here are some ideas to get you started:
  
=== SMB2 protocol improvements ===
+
=== SMB3 protocol improvements ===
*The SMB2 protocol (followon to cifs) adds many useful performance enhancements and new features.  The Linux kernel implementation is still experimental and is missing key features including SMB2.1 **dialect support (items such as lease keys) and lacks a useful credit request algorithm (which Samba server only recently added for the server side). Various performance optimizations (including **support for very large reads and writes and dispatch of more requests in parallel) are also possible.
+
*The SMB2 protocol (followon to cifs) and now the SMB3 protocol, new to Windows 8, Windows 2012 and Samba 4, add many useful performance enhancements and new features.  SMB2.1 support, and even minimal SMB3 support, is merged into the Linux kernel client since the 3.8 kernel, but there are many useful optional features still to be implemented.  A sample list of possible features to code includes:
 +
* Support for using multiple network interfaces at once under the same mount (SMB3 multichannel)
 +
* Improved directory and metadata caching ("directory oplocks")
 +
* Improved failover in clustering environments ("persistent file handles" and more generally SMB3 "continuous availability" support)
 +
* File copy offload (there are multiple server side copy mechanisms possible with SMB3, T10 copy offload, SMB2/SMB3 "copy chunk" as well as an older cifs mechanism "SMB Copy" that could be implemented, compared and optimized).  This is especially timely given the improvements in Samba/btrfs integration which better optimizes SMB2/SMB3 "copy chunk" handling on the server.   This could improve file copy performance by orders of magnitude.
 +
* HA availability improvements on server and/or client (the SMB3 "Witness protocol")
 +
*
 
* Language: C
 
* Language: C
* Difficulty: Varies, Medium to Hard
+
* Difficulty: Varies, Medium to Hard, but most of the protocol features are at least well documented in WSPP and have implementations in current Windows clients and servers already
 
* Possible Mentors: Steve French
 
* Possible Mentors: Steve French
  

Revision as of 22:08, 28 March 2013

Google Summer of Code: Suggested Project ideas

The following are the Samba project ideas for Summer of Code. Of course you are free to come up with ideas not listed here. Please discuss the your planned project by either joining us on irc://irc.freenode.net/#samba-technical or by sending email to samba-technical@samba.org

Most of our projects will require C programming skills, but the Samba4 section has a couple of Python projects.

Samba 3

There are currently no project proposals for this area.

Samba 4

Some additional possible GSoC topics can be found in Bugzilla in the form of bugs which are marked as "Feature request": here. Questions regarding complexity and requirements should be directed to the technical mailing list.

Admin Utilities

We still need a few more Samba4-specific administration utilities to make Samba4 useful in real life. A Summer of Code student would be expected to do a number of these (identifying new needs from deployments), but here are some suggestions:

Setup / provision GUI for Samba (and OpenChange)

Setting up Samba and OpenChange currently requires running quite a lot of command line utilities. It should be easy for even a junior system administrator to set up Samba as a Domain Controller, RODC and general server.

The windows equivalent tool is dcpromo.exe, and while you may not want/need to copy it directly, it does provide a reasonable basis for establishing what this tool should be able to provide (in terms of outcomes).

Some initial work (using PyQt4) is available at http://gitorious.org/samba-openchange-gui-tools/samba-openchange-gui-setup but working with that is not essential to this task.

  • Difficulty: Easy
  • Language(s): Python
  • Possible mentors: Brad Hards, probably others.

Windows Search Protocol WSP client library and torture tests

The Windows Search Protocol WSP is used to implement remote full filesystem indexing (indexed search) between windows machines. We would like to support this functionality in Samba, interfacing with existing indexing tools on Unix systems (such as beagle).

This is a new protocol based on SMB named pipes \pipe\ci_skads or \pipe\MSFTEWDS. See http://msdn2.microsoft.com/en-us/library/cc216195.aspx.

The student should write a (un)marshalling library to push and pull PDUs and an async client library on top of the samba4 raw smb client library.

The student should write sub tests for smbtorture which should demostrate how the protocol works against a windows server.

The student doesn't have to implement the samba4 server code.

  • Difficulty: Hard
  • Language(s): C, (Python?)
  • Possible Mentors: Tridge

Browsing support in Samba 4

Samba 4 still needs support for mailslots in general and in particular for the BROWSE mailslot. Should come with tests. Documentation of the BROWSER protocol is available here: http://msdn.microsoft.com/en-us/library/cc201609(PROT.10).aspx http://ubiqx.org/cifs/Browsing.html

  • Difficulty: Hard
  • Language(s): C
  • Possible mentors: JelmerVernooij, Stefan Metzmacher

Make samba 4 DC Group Policies (GPO) aware

Currently Samba 4 DC is able to serve GPOs to clients and they are mostly able to act according to the content of those GPOs. But even if the GPO concerns AD DCs, Samba 4 ignore them even if some parameters are meaningful in a Samba 4 context (ie. password length, password life ...).https://wiki.samba.org/index.php?title=SoC/Ideas&action=edit&section=9 In order to work around this limitation, there is currently a couple of scripts that allow to set them but it's a suboptimal experience.

The goal of this project is to make Samba 4 periodically check if there is a GPO for it, check if any parameter of this GPO are meaningful for Samba (as a counter example a GPO which defines the background color of the Desktop on DC is not meaningful for Samba 4) and if so to alter parameters accordingly. More details can be found in Matthieu's samba-technical email.

  • Difficulty: Easy, Medium
  • Language(s): C, Python
  • Possible mentors: Matthieu Patou

Implement login / logout related counter update

For the moment the attributes related to login and logout are not updated by Samba4. The goal of this project is to understand in which case windows update the counters (ie. most probably during interactive logon but also maybe with some netlogon calls ?) and to implement counter and timestamp update is Samba code so that this information can be available. This project of course includes the development of unit tests.

  • Difficulty: Easy
  • Language(s): C
  • Possible mentors: Andrew Bartlett

Linux Kernel CIFS/SMB2 client improvements

Interested students should contact Steve French or Jeff Layton and discuss possible improvements to the Linux Kernel CIFS VFS client. Here are some ideas to get you started:

SMB3 protocol improvements

  • The SMB2 protocol (followon to cifs) and now the SMB3 protocol, new to Windows 8, Windows 2012 and Samba 4, add many useful performance enhancements and new features. SMB2.1 support, and even minimal SMB3 support, is merged into the Linux kernel client since the 3.8 kernel, but there are many useful optional features still to be implemented. A sample list of possible features to code includes:
  • Support for using multiple network interfaces at once under the same mount (SMB3 multichannel)
  • Improved directory and metadata caching ("directory oplocks")
  • Improved failover in clustering environments ("persistent file handles" and more generally SMB3 "continuous availability" support)
  • File copy offload (there are multiple server side copy mechanisms possible with SMB3, T10 copy offload, SMB2/SMB3 "copy chunk" as well as an older cifs mechanism "SMB Copy" that could be implemented, compared and optimized). This is especially timely given the improvements in Samba/btrfs integration which better optimizes SMB2/SMB3 "copy chunk" handling on the server. This could improve file copy performance by orders of magnitude.
  • HA availability improvements on server and/or client (the SMB3 "Witness protocol")
  • Language: C
  • Difficulty: Varies, Medium to Hard, but most of the protocol features are at least well documented in WSPP and have implementations in current Windows clients and servers already
  • Possible Mentors: Steve French

Support for SELinux

  • Mac Security Label support is important for virtualization and useful for improved security some workloads. Support for setting/getting these labels over the wire was investigated in the NFS version 4 workgroup. Adding support to the CIFS Unix Extensions (Linux kernel client and Samba server) should be possible, especially if this is just a new class of extended attribute. The goal would be to support this feature of SELinux to allow KVM and other applications to take advantage of security labels. Some of the background requirements are loosely related to the (nfs equivalent of) what is mentioned in: http://tools.ietf.org/html/draft-quigley-nfsv4-sec-label-01
  • Language: C
  • Difficulty: Hard
  • Possible Mentors: Steve French

Create GUI or command-line tools for displaying /proc/fs/cifs statistics and and mount/session status

  • might also involve some cleanup of the in-kernel stats / status output
  • Language: some C (for kernel code), something else for GUI?
  • Difficulty: Easy
  • Possible Mentors: Steve French

Create a common uid mapping mechanism for Linux nfs and cifs vfs clients

  • or maybe just figure out a way to hook cifs up to rpc.idmapd
  • add a way for the client to remap the uids returned by the server to uids which would be valid on the client (or to a default if such uid does not exist).
  • This is helpful especially when the server supports the CIFS Unix Extensions and has different uids and gids mapping than the client
  • Difficulty: Hard
  • Possible Mentors: Steve French

VFS change notification support

  • add VFS support for calling into the filesystem when setting up notifications
  • add code to cifs/smb2 to set up and deal with notifications from the server in response to inotify/dnotify calls
  • Difficulty: Hard
  • Possible Mentors: Steve French

Support for retrieving snapshots, encrypted files, or compressed files from Windows

  • Difficulty: Medium
  • Possible Mentors: Steve French

cifs->Samba automated test facility

  • Do build verification similar to what we can now do with the Samba server and tools in the Samba build farm. Mounts from the Linux SMB2 and CIFS kernel clients could be tested with posix file i/o tests which might include modified versions of the "connectathon" and xfstest test suites and others. The goal is to quickly identify problems with newly integrated patches.
  • Difficulty: Hard
  • Possible Mentors: Steve French

Other Random Ideas

  • Ideas aren't limited to these, feel free to propose something else:
    • Create a GUI for creating and managing Linux cifs mounts, and more easily configuring the many complex cifs mount options, statistics (/proc/fs/cifs)
    • Support for alternate transport protocols (other than TCP sockets). Adding support for SCTP to cifs/smb2 kernel clients and Samba server or perhaps more interesting add support for Linux's "virtio" transport to the cifs/smb2 kernel clients and Samba server (to allow optimized mounts and zero-copy transfer of data from virtualized guests to hosts on the same box)
    • Support for features (such as directory delegations) which NFS version 4.1 has but which current CIFS even with the most current CIFS->Samba protocol extensions (CIFS Unix Extensions) do not have -- will probably need server support too.
    • Add additional library support or modify Samba client libraries so they can use existing kernel cifs functions (such as sending SMBs on negotiated sessions when the kernel client already has a session to the server). With the addition of library to access cifs's pipe (in kernel), Samba client libraries or other dce/rpc code could use cifs kernel sessions for management of and over cifs mounts.
    • Add libraries and utilities to manage acls (cifs kernel client has an extended attribute for setting/getting "raw" cifs acls but userspace posix acl tools obviously can't be used to manage cifs specific acl features).
  • Difficulty: Varies
  • Language(s): C
  • Possible mentors: Steve French

Build Farm

The [Build Farm] is a set of machines with different configuration that regularly rebuild the latest snapshots of Samba and other projects on different platforms, to catch portability issues. It has a web interface and sends out emails.

Improve Build Farm look and Feel

Samba's build farm still hasn't adopt the new Samba graphical chart and the look and feel is not very good. With this submission we propose to address this with the following objectives:

  • Main ideas:
    • Adopt the new samba style
    • Improve reporting (ie. present which are the build that can't be built and which are not, daily emails, ...)
    • Make test errors quickly accessible, in this example, user has to scroll a long time before meeting the errors
    • Add the capacity to manage flaky tests, reduce emails alerts (ie. need 2 consecutive builds with the same flacky test to trigger a real error)
    • Improve page loading speed (ajax ?)
  • Difficulty: Easy to Medium
  • Language(s): HTML, CSS, Python
  • Possible mentors: Matthieu Patou, JelmerVernooij

Samba GTK+

Samba-GTK Samba-GTK is a set of GTK+ frontends for Samba written in Python.

Extension of the GTK+ frontends

There are a couple of GTK+ frontends for Samba4 (see SambaGtk). These are very limited at the moment but you could work on expanding them and further integrating them with GNOME. Language: C or Python

  • Difficulty: Easy
  • Language(s): Python, perhaps C
  • Possible mentors: JelmerVernooij

Port to GTK3