Using Git for Samba Development

From SambaWiki
Revision as of 02:57, 30 July 2019 by Abartlet (talk | contribs) (link to autobuild)

Overview

Samba source code is now hosted in a shared git repository to which all existing Samba Team members are allow to push to after code review via the autobuild system. If you are a Team member and do not have an account on git.samba.org, please email the team list and let the appropriate people know. This is a shared repository, after review and via autobuild, all active development is in master as a series of rebased patches. Samba's use of git is perhaps best described as the Centralised, rather than the Feature branch git workflow. Individual commits should be kept small and should be summarized completely by their commit message.

Git Installation

The examples in the following sections are based off of the tools and syntax used by git v1.5.3 or later which is the recommended version for Samba. Either apt-get, yum, or make install the tools. See Git documentation for more details on this part.

If you are on OS X, both MacPorts and Fink contain working binary packages of Git. Building Git from source on OS X works fine, but resolving the chain of dependencies for asciidoc (which is needed to build the man pages) is very painful.

GitLab

Many users may find it easier to start by creating a fork on GitLab. Samba has a GitLab mirror of the official git repository.

A merge request made against the GitLab mirror will send an email to everyone registered on GitLab with your patch. You may wish to also mail the patch or merge request URL to the samba-technical mailing list. Please ensure you have added Signed-off-by tags to your commits, and follow the Samba copyright policy.

Once on the mailing list, Samba Team members (in particular) will review your changes, and post comments to you either on your merge request, or on the list. so please ensure you are subscribed to samba-technical.

To fetch a merge request from gitlab, use the fork URL in the mail, giving the original branch:

git fetch git@gitlab.com:<fork> <branch>
git checkout FETCH_HEAD

Once Reviewed-by tags are added this can then be sent to autobuild in the customary fashion.

After the autobuild completes, please close the merge request using the git hash finally assigned in master in the comment:

Merged as <git hash>


Automatic testing of commits

The GitLab CI service is configured in the Samba repository. When you create a commit Samba will be automatically built and some of Samba's tests will be run. GitLab tests on shared runners are prone to failure due to GitLab's limit of one hour on tests. To help prevent this a smaller set of tests is run on the GitLab runners as the full set of tests can take upwards of 4 hours. Members of the Samba team and approved Samba developers have access to a set of private servers to run tests on which you can read more about here. A guide on setting up a personal computer as a GitLab runner can be found here.

Basic Samba Git

The master git repository is located at git://git.samba.org/samba.git and https://git.samba.org/samba.git. There is also a GitWeb installation.

Step Zero is to set your name and email address for commits:

 $ git config --global user.email Your.Email@domain.com
 $ git config --global user.name "Your Real Name"

Next, clone the repository:

 $ git clone https://git.samba.org/samba.git
 Initialized empty Git repository in /home/AD/gcarter/src/git/tmp/samba/.git/
 remote: Generating pack...
 remote: Done counting 440544 objects.
 remote: Deltifying 440544 objects...
 remote:  100% (440544/440544) done
 Indexing 440544 objects...
 remote: Total 440544 (delta 340808), reused 436199 (delta 336480)
  100% (440544/440544) done
 Resolving 340808 deltas...
  100% (340808/340808) done
 $ cd samba

List local and remote branches:

 $ git branch
   * v3-2-test
 $ git branch -r
   origin/HEAD
   origin/v3-0-stable
   origin/v3-0-test
   origin/v3-2-stable
   origin/v3-2-test

Listing tags matching release-3-0-3*

 $ git tag -l release-3-0-3*
 release-3-0-3
 release-3-0-3pre1
 release-3-0-3pre2
 release-3-0-3rc1

Creating your own working branch from v3-2-test:

 $ git checkout -b my_branch origin/v3-2-test
 Branch my_branch set up to track remote branch refs/remotes/origin/v3-2-test.
 Switched to a new branch "my_branch"

Do your own local work:

 $ mkdir git-test
 $ echo "hello" > git-test/README

View status of changes

 $ git status
 # On branch my_branch
 # Untracked files:
 #   (use "git add <file>..." to include in what will be committed)
 #
 #       git-test/
 nothing added to commit but untracked files present (use "git add" to track)

Add our new file to the local branch index:

 $ git add git-test
 $ git status
 # On branch my_branch
 # Changes to be committed:
 #   (use "git reset HEAD <file>..." to unstage)
 #
 #       new file:   git-test/README
 #

Commit changes

 $ git commit -m "Example file for HOWTO"
 Created commit ad9a1eb: Example file for HOWTO
  1 files changed, 1 insertions(+), 0 deletions(-)
  create mode 100644 git-test/README

Do some more work and commit local changes....

Now fetch the remote branch history:

 $ git fetch

Merging remote branch changes:

 $ git merge origin/v3-2-test
 Already up-to-date.

To present your patchset properly to other developers, please rebase your patches against the branch you are developing against. In particular, if you are developing for Samba 3, please regularly do a

 $ git rebase origin/v3-2-test

Obviously replace "origin/v3-2-test" by "origin/master", "origin/v3-4-test" or whatever branch you are developing against. If you have created a mess in your local patch queue, "git rebase -i" might help you out.

The final push of the changes to the shared repository is done by team members using the autobuild system.

However, you may push them to your own fork on gitlab or elsewhere with a command like:

 $ git remote add gitlab ssh://git@gitlab.com/$USER/samba
 $ git fetch gitlab
 $ git push gitlab my_branch:my_remote_branch
 updating 'refs/heads/v3-2-test' using 'refs/heads/my_branch'
   from f2252e041e075e19bf27e53ab3ed62f39bc8b3e2
   to   ad9a1eb599c125964ac3e198d3003841edb4c54e
 Generating pack...
 Done counting 5 objects.
 Result has 4 objects.
 Deltifying 4 objects...
  100% (4/4) done
 Writing 4 objects...
  100% (4/4) done
 Total 4 (delta 1), reused 0 (delta 0)
 refs/heads/v3-2-test: f2252e041e075e19bf27e53ab3ed62f39bc8b3e2 -> ad9a1eb599c125964ac3e198d3003841edb4c54e

After they have been pushed to your GitLab fork make a merge request for the branch you are developing against. Once it has been reviewed an approved a team member will push them to the shared repository with the autobuild system as normal.

Some Git Tricks

When you have quite some changes to commit, but you want to split them in logically coherent commits you can do that with the following invocation:

 $ git commit -i

When you have commits you want to push, but you want to have the commits to compile on their own, you can insert x make -j after each commit to check if they do compile one after the other with:

 $ git rebase -i

Last, but not least, when you want to discard part of your changes before committing, you can use:

 $ git reset -p

Explanation of Branches

  • The master branch is the development branch. Any team member may check into this branch via autobuild after code review.
  • The *-test branches are the release series dev branch. These are under the control of the current release manager for a given release series.
  • The *-stable branches are used for release purposes, matching the current release tarball. These are under the control of the current release manager for a given release series.

The Samba Release Planning page lists the current branches and their status.

Creating patches for the samba-technical mailing list or Bugzilla

As you need code review you will first need to send your patches to other developers. GitLab provides a one-step approach, but we also accept patches on the samba-technical mailing list and in Bugzilla.

In this case, you should create patches. Now, assuming your patches are the last three commits on your current local git branch, this is the easiest way to create patches from them:

This will export them into one file (which is the preferred way to handle patches in Samba):

 $ git format-patch -3 --stdout  > featureX-01.patches.txt

This will create a patch file with a .txt suffix (meaning it will be encoded text/plain in the archives) in your current directory that you can then send to the samba-technical mailing list.

While there are many ways to send patches with git, a single .txt file attached to the cover e-mail is how we do things in Samba.

Note that if you have a number of patches against a specific samba.git branch and don't feel like counting them, the following works as well (e.g. against the master branch):

 $ git format-patch --stdout origin/master > master-featureX-01.patches.txt

This will export all the changes since origin/master into one file.

git send-email

Using git send-email in the Samba project is not permitted. Doing so will cause, without manual work by the recipient, your patch to be credited to samba-technical@... not your own e-mail address.

FAQ

Q. How do I revert a commit?

A. The "git reset" command allows you to reset the HEAD of the branch to any given point in history. To go back one commit, run "git reset HEAD^". This will keep your local changes and you can make any additional changes before re-commiting the new work. Also see the "git commit --amend" command and the "git reset" man page for other examples.


Q. Is there a shorthand for git push <URL> <local_repo:remote_repo>?

A. Yes. You can define a [remote "upstream"] section in your local repos .git/config file.

 $ git config remote.origin.url ssh://git.samba.org/data/git/samba.git
 $ git config remote.origin.push my_branch:v3-2-test
 $ cat .git/config
 ...
 [remote "origin"]
         url = ssh://git.samba.org/data/git/samba.git
         fetch = +refs/heads/*:refs/remotes/origin/*
         push = my_branch:v3-2-test
 [branch "my_branch"]
         remote = origin
         merge = refs/heads/v3-2-test


Q. How can I have access to the old CVS and SVN imported repositories via GIT without creating additional cloned repos?

A. You can add remote tracking repositories using "git remote":

 $ git remote add cvs git://git.samba.org/import/samba-cvsimport.git
 $ git fetch cvs
 remote: Generating pack...
 remote: Done counting 7070 objects.
 Result has 5004 objects.
 remote: Deltifying 5004 objects...
 remote:  100% (5004/5004) done
 Indexing 5004 objects...
 remote: Total 5004 (delta 3836), reused 4098 (delta 3044)
  100% (5004/5004) done
 Resolving 3836 deltas...
  100% (3836/3836) done
 461 objects were added to complete this thin pack.
 * refs/remotes/cvs/APPLIANCE_HEAD: storing branch 'APPLIANCE_HEAD' of git://  git.samba.org/import/samba-cvsimport
   commit: af0e201

You can then see the remote branches using "git branch -r" and work with them in the same way you did for the "origin" branches obtained in the initial "git clone".

Q. How can I maintain a feature branch against the upstream Samba branches?

A. You clone the Samba repository as per the instructions above. Then you make a new feature branch from v3-2-test:

$ git branch feature/foo v3-2-test

Now you do your development in your feature branch. Any time the v3-2-test branch gets too different to the code in your feature branch you should rebase your feature branch. The rebase rewinds your feature branch to the point there it was branched. Then it updates you feature branch to the HEAD of the v3-2-test branch and re-applies your changes.

$ git rebase v3-2-test
First, rewinding head to replay your work on top of it...
HEAD is now at 357f003... Add WERR_SERVICE_ALREADY_RUNNING.
...
Wrote tree 02299ef7c1cfa093248bfd9c6e3812805718845e
Committed: e20a8c521d7860d9b7bd724ed5ea19c7306530ab

Rebase works best when you use it for local branches that are never pushed to a public repository, see Why won't "git push" work after I rebased a branch?.

Suggestions for .gitconfig

In order to comply with Samba coding standards and not add new trailing whitespace (see: Samba Coding Standards Document) it is suggested to put:

[apply]
       whitespace = strip

into your $HOME/.gitconfig file.

That way, whenever you apply a patch using git-am or rebase your work on top of the upstream trees with git-rebase, all newly added whitespace is nicely removed automatically.