Testing development trees

Testing development trees

This is a copy of an email I sent to the team list a while ago on how to semi-automate the process of testing Samba patches against both the s3 and s4 testsuite. This should be done before patches are proposed for the master tree.

Hi Folks,

I thought it might be useful to share experiences on how we prevent breaking the build/test of s3/s4, especially with the merged tree. This came out of a discussion on IRC with Steven and Andrew.

The approach I use is this:

  • a main git devel tree. This is where I commit from, do my day to day development etc. e all have one of these. Let's assume mine is in /home/tridge/samba/devel
  • a s4-test tree. This was created like this:
cd /home/tridge/samba
git clone /home/tridge/samba/devel s4-test
  • a s3-test tree. Created similarly, like this:
cd /home/tridge/samba
git clone /home/tridge/samba/devel s3-test

The key is that the s3-test and s4-test trees have my devel tree as their upstream. This means "git pull" in /home/tridge/samba/s3-test pulls changes from the master branch of my devel tree.

I then have this script sitting in a common directory:

 git pull && ./autogen.sh && ./configure.developer && make clean && make && make test
 cp summary summary.$(git log|head -1|cut -d' ' -f2)

I call that script "rebuild.sh"

Ok, so I have some changes in my devel tree that I want to test with s3 and s4 before I push. I do this:

  • in another window, I cd to s3-test/source3 and run rebuild.sh
  • in a 3rd windows I cd to s4-test/source4 and run rebuild.sh

(I run these in screen, with logging on, so I can review any errors easily).

While that is happening I can keep developing with my main tree, usually on a branch. The running of the two tests takes about an hour in my laptop. When it passes both tests (or only has expected failures) I then know my master devel branch is OK to push.

I then do a final git pull, and rebase if needed in case someone else has pushed changes in the mean time. If that has happened and the changes seem likely/possible to conflict I go back and re-run the full testing. At minimum I run "make quicktest" in s4 to make sure the build and basic tests don't conflict with other peoples changes from the last hour. Otherwise I push.

To work out what failures I should be expecting in s3/s4 at any one time I tend to check the more reliable hosts on the build farm (eg. svart, tridge, trip, opi, burns etc).

The above certainly isn't foolproof, but it's better than nothing.

Anyone else like to share their approach? Perhaps once we work out what the "best practice" is we could put this on the developer wiki?

Cheers, Tridge