SoC/Ideas: Difference between revisions

From SambaWiki
(add new freebsd CI task)
Line 122: Line 122:
* Language(s): Some knowledge of C
* Language(s): Some knowledge of C
* Possible Mentors: Aurélien Aptel
* Possible Mentors: Aurélien Aptel

<!-- Commented out possibly stale proposals


===Samba AD DC as the ideal POSIX Directory===
===Samba AD DC as the ideal POSIX Directory===
Line 138: Line 136:
*Languages(s): C, Python
*Languages(s): C, Python
*Possible Mentors: Andrew Bartlett
*Possible Mentors: Andrew Bartlett


===GitLab CI of Samba for non-linux platforms (FreeBSD in particular)===

Samba uses GitLab CI to improve the quality of our patches. Efforts are currently underway to extend the docker container used from just Ubuntu 14.04 to later versions and other distributions.

However, we occasionally have issues ensuring Samba still builds and operates on FreeBSD and other non-linux platforms.

The idea would be to have a docker image and .gitlab-ci.yml code to support it that runs FreeBSD and then builds and runs Samba's testsuite inside that FreeBSD nested VM, while still outputting the results to the normal gitlab-ci.

This differs from just running GitLab CI runners on FreeBSD as we need auto-scale, destroy the host and guest at the end of the test and run on Linux docker (such as the free GitLab,com CI runners).

As a stretch goal, being able to run some tests against a specific linux kernel and a raw ext4 filesystem (rather than unionfs) via qemu rather than depending on the docker host configuraiton.

*Difficulty: Medium
*Language(s): Python, shell, YAML
*Possible Mentors: Andrew Bartlett


<!-- Commented out possibly stale proposals


===Make libsmbclient thread-safe for Gnome VFS===
===Make libsmbclient thread-safe for Gnome VFS===

Revision as of 20:26, 23 January 2019

Google Summer of Code: Suggested Project ideas

The following are the Samba project ideas for Summer of Code. Of course you are free to come up with ideas not listed here. Please discuss the your planned project by either joining us on irc://irc.freenode.net/#samba-technical or by sending email to samba-technical@lists.samba.org

Samba

Some additional possible GSoC topics can be found in Bugzilla in the form of bugs which are marked as "Feature request": here. Questions regarding complexity and requirements should be directed to the technical mailing list.

Print System Asynchronous Remote Protocol Wireshark Dissectors

The Print System Asynchronous Remote Protocol (MS-PAR) is a replacement for the synchronous Print System Remote Protocol (MS-RPRN). MS-PAR inherits many message and buffer formats from the old protocol, but allows for asynchronous submission and notification of print jobs. Further details of the protocol can be found in Günther and Andreas' SambaXP presentation.

The student should write Wireshark dissectors for MS-PAR. The student should improve existing smbtorture tests, which demonstrate how the protocol works against a Windows server.

  • Difficulty: Medium
  • Language(s): C
  • Possible Mentors: Andreas Schneider


dbwrap back-end for Ceph RADOS key-value storage

Ceph offers a highly scalable and fault-tolerant storage system. Samba is already capable of sharing data located on the Ceph Filesystem, however scale-out sharing (the same data exposed by multiple Samba nodes) currently requires the use of CTDB for consistent and coherent state across Samba cluster nodes. In such a setup CTDB provides a clustered database with persistent key-value data storage and locking. Database usage is abstracted out via a generic dbwrap interface.

Ceph's librados library provides an API for the storage and retrieval of arbitrary key-value data via the omap functions. A watch/notify protocol is also provided as a mechanism for synchronising client state (locking). Key-value data stored in the RADOS back-end inherits the same redundancy features as regular objects, making it a potentially good candidate as a replacement for CTDB in scale-out Samba clusters.

This task involves the implementation and testing of a new dbwrap back-end that uses librados for the storage, retrieval and locking of Samba key-value state. Ideally, the candidate would also allow time for benchmarking.

  • Difficulty: Medium
  • Language(s): C
  • Possible Mentors: David Disseldorp


Samba binary size reduction

Samba has grown to quite a bloated beast. This task will focus on some areas where the bloat can easily be reduced, e.g.

  • Removal of unused autogenerated librpc code
    • RPC client and server code when only serialization functions are used
    • optional struct print routines (pidl noprint?)
    • Some knowledge of perl would help here (for pidl)
  • Add new build options to compile Samba without certain functionality
    • Undesired DCE/RPC services
    • Printing support
    • Legacy SMB/CIFS support (stretch goal)


  • Difficulty: easy
  • Language(s): Some knowledge of C and python would be helpful
  • Possible Mentors: David Disseldorp (supported by Andrew Bartlett)

Fuzz Samba using American Fuzzy Lop

Try to find bugs in the server or client components of Samba using AFL. This is not the usual AFL scenario where you can fuzz a file format parser. It needs to be a bit smart about it, as the way the SMB protocol works, you need to do a series of steps before reaching arbitrary SMB commands (protocol negotiation, session setup, tree connect, ...). Similarly some checks like packet signing should be worked around to reach the best results from AFL.

Samba code will need to be modified in hackish ways to make this work. I don't expect to be able to merge it back but if some of the modification required could be cleaned up and integrated in would be a bonus.

  • Difficulty: medium
  • Language(s): Some knowledge of C
  • Possible Mentors: Aurélien Aptel

Samba AD DC as the ideal POSIX Directory

Samba is a great Active Directory Domain Controller, but it is not an ideal directory server for a large, passionate and important user base: Sites with Samba SMB servers, but also general purpose Linux servers. A smaller subset of these sites also have Linux desktops. These sites may also have Windows servers, but they like the Windows desktops, are not the focus.

These sites often used Samba + OpenLDAP, and are finding the move to Samba's AD DC a bit difficult, because schema extension is hard, some things are not done automatically (like uidNumber allocation), and in general the focus has been around matching Windows not listening to the needs of this part of our user base.

Specific research should be done into what FreeIPA does well in targeting this user segment, and what customisations advanced users of OpenLDAP apply.

This project would be to propose a number of specific improvements, and to add both tests and an implementation of these improvements to Samba.

  • Difficulty: Hard
  • Languages(s): C, Python
  • Possible Mentors: Andrew Bartlett


GitLab CI of Samba for non-linux platforms (FreeBSD in particular)

Samba uses GitLab CI to improve the quality of our patches. Efforts are currently underway to extend the docker container used from just Ubuntu 14.04 to later versions and other distributions.

However, we occasionally have issues ensuring Samba still builds and operates on FreeBSD and other non-linux platforms.

The idea would be to have a docker image and .gitlab-ci.yml code to support it that runs FreeBSD and then builds and runs Samba's testsuite inside that FreeBSD nested VM, while still outputting the results to the normal gitlab-ci.

This differs from just running GitLab CI runners on FreeBSD as we need auto-scale, destroy the host and guest at the end of the test and run on Linux docker (such as the free GitLab,com CI runners).

As a stretch goal, being able to run some tests against a specific linux kernel and a raw ext4 filesystem (rather than unionfs) via qemu rather than depending on the docker host configuraiton.

  • Difficulty: Medium
  • Language(s): Python, shell, YAML
  • Possible Mentors: Andrew Bartlett


Linux Kernel SMB Client Improvements

The Linux Kernel has a module called cifs.ko which is independent from Samba (it doesn't share code) that allows users to mount remote shares. It supports multiple dialects of SMB (1, 2, 3). The protocol dialects are now officially documented by Microsoft (See MS-SMB, MS-SMB2) so students shouldn't have to worry about reverse engineering to understand them. The Wireshark open source network sniffer&dissector is a very good learning tool as well.


Interested students should contact Steve French or the linux-cifs mailing list to discuss possible improvements to the Linux Kernel CIFS VFS client.

Add machine-readable debug & stats /proc file

  • We currently output debug and statistic information under /proc/fs/cifs/ (DebugData, Stats, ...). We need to stop outputing free format text that breaks all parsers out there everytime we add things to it. Clean up the cifsdebug.c file (it is kind of messy). Possibly generate a hierarchy of /proc files (e.g. a dir per tcp connection, subdirs for session, files for tcons) instead of dumping everything in one file.
  • Make a nice visualizer/dashboard thing to get an overview. This could be console or GUI or...
  • Language: C for the kernel stuff, Userspace can be C, C++, Python.
  • Difficulty: Low

Add additional ftrace (trace-cmd) tracepoints and better GUI

  • Add more dynamic tracepoints to cifs.ko for commonly needed user scenarios, and add GUI (and/or CLI) tool to make it easier to enable/disable cifs.ko dynamic trace points (See /sys/kernel/debug/tracing/events/cifs/ for the pseudo-files that are currently configured manually for tracing or via trace-cmd) Make a nice native/console/web UI for it.
  • Language: C (any kernel changes) and userspace C, C++ or Python
  • Difficulty: Low

Add performance analysis cli tools

  • Add more perf tools for SMB3 client (similar to iostat or nfsstat) that leverage (and possibly extend what is captured) in /proc/fs/cifs/Stats but make it easier to analyze performance of a cifs mount
  • Language: C, C++ or Python
  • Difficulty: Low

Improve smbcmp, the capture diff tool

  • smbcmp is a simple CLI tool that uses Wireshark CLI version (tshark) to dump and diff traces. It currently uses the plain text output format of tshark but tshark also has a proper XML output. The goal of this project would be to use or combine current tshark output with the XML output to do better and deeper diffs (ignoring indentation differences, adding ways to let users add ignore rules, etc).
  • Make smbcmp highlight diffs from the packet summary listing (currently it only diffs the "detail packet" output).
  • These are just ideas, students are free to propose and work on their own ideas.
  • Language: Python (a rewrite in something else is OK too)
  • Difficulty: Low

Write the One-True-Tool to unify probe/setup/configuration cifs.ko properly

  • There are too many knobs in different places at the moment: request-keys, idmap, cifscreds, /proc stuff. This goal of this project would be to write one CLI tool that would wrap everything under a common interface. It could handle getting/setting ACL as well.
  • This would be a userspace project.
  • The implementation shouldn't too hard but the student will need to learn about the existing configuration ways and usecases which can be long.
  • Language: C, C++, Python
  • Difficulty: Medium

Add support for DAC (Claims Based ACLs) to the smb3 kernel module and tools

  • Similar to what was done to extend the Apache access control model to allow richer ACL semantics,
  • SMB3's access control model was extended (at least in Windows). See e.g. Dynamic Access Control.
  • Add support to the SMB3 kernel module and user space tools (or create new ones) to allow viewing and managing claims based ACLs (DAC) from the client.
  • Difficulty: Medium
  • Language: C (kernel), C/C++/Python (user space tools)

Add support for ODX (T10) Copy Offload to the smb3 kernel module

  • Windows, and various NAS servers support ODX copy offload (e.g. NetApp),
  • to allow much faster server side copy.
  • Add support to the kernel client for this and integration with existing or new copy offload tools to make it easy to use.
  • For more information see MS-SMB2 and SMB3 offload data transfer and Offloaded data transfers and references at bottom of Server-Side_Copy.
  • Language: C
  • Difficulty: Medium/High

VFS change notification support in cifs.ko

  • The kernel provides a file/dir notification API (inotify, dnotify). The SMB protocol also provides a way to get notified of file changes. This project would be about implementing the inotify API for cifs.ko by making use of the SMB notification mechanism.
  • Difficulty: Hard
  • Language: C

Failover/Continuous Availability and HA improvements (Witness protocol)

  • Benefits: Improved reliability, data integrity - may also allow planned migrations (moving data from one server to another).
  • Challenges: Complexity, requires additional RPC infrastructure in client. There is a Samba user space prototype of the Witness protocol that could be reused (since we only need the client part of the RPC calls).
  • Language: C
  • Difficulty: High