Samba on GitLab: Difference between revisions
(add service status info) |
(link back to better instructions in new contribute page) |
||
(40 intermediate revisions by 4 users not shown) | |||
Line 1: | Line 1: | ||
Samba has a [https://gitlab.com/samba-team/samba GitLab mirror] of the [https://git.samba.org/samba.git official git repository] used to aid development |
Samba has a [https://gitlab.com/samba-team/samba GitLab mirror] of the [https://git.samba.org/samba.git official git repository] used to aid development |
||
=Where is |
=Where is Samba on GitLab?= |
||
https://gitlab.com/samba-team |
* Samba's Official GibLab mirror is at https://gitlab.com/samba-team/samba |
||
* The Samba shared CI repository is at https://gitlab.com/samba-team/devel/samba |
|||
=Getting Access= |
=Getting Access= |
||
Line 15: | Line 17: | ||
Just ask on the team list from your team e-mail with your gitlab username. |
Just ask on the team list from your team e-mail with your gitlab username. |
||
A fellow team member with '''Maintainer''' rights can then add you a '''Developer''' via this page: |
A fellow team member with '''Maintainer''' rights can then add you as a '''Developer''' via this page: |
||
* [https://gitlab.com/groups/samba-team/-/group_members Samba team group members] |
* [https://gitlab.com/groups/samba-team/-/group_members Samba team group members] |
||
Line 23: | Line 25: | ||
Our broader developer community is encouraged to apply to use the Samba CI repo. Access is given to folks active on the mailing list with patches to submit, not just to Samba Team members. |
Our broader developer community is encouraged to apply to use the Samba CI repo. Access is given to folks active on the mailing list with patches to submit, not just to Samba Team members. |
||
Please write a mail to contributing@samba.org with details of your contribution history or plans and a Samba Team member with '''Maintainer''' rights can then add you as a '''Developer''' via this page: |
Please write a mail to contributing@samba.org '''with your GitLab username''', details of your contribution history or plans and a Samba Team member with '''Maintainer''' rights can then add you as a '''Developer''' via this page: |
||
* https://gitlab.com/samba-team/devel/samba/project_members |
* https://gitlab.com/samba-team/devel/samba/project_members |
||
Line 37: | Line 39: | ||
==Code of conduct== |
==Code of conduct== |
||
Please prefix branches with your username, and be nice. Don't overwrite the work of others. |
Please prefix branches with your gitlab username, and be nice. Use only to develop Samba. Don't overwrite the work of others. |
||
$USER/topic |
|||
eg |
|||
⚫ | |||
frednurk/kerberos-timeout-handling |
|||
In return you get a full CI run using Samba Team provided resources running thanks to a credit in Rackspace's cloud. That in turn makes it easier for Samba Team members doing [[CodeReview|Code Review]] as your patches will work first time, and they can see proof of that. |
In return you get a full CI run using Samba Team provided resources running thanks to a credit in Rackspace's cloud. That in turn makes it easier for Samba Team members doing [[CodeReview|Code Review]] as your patches will work first time, and they can see proof of that. |
||
If you describe your work in the branch name, this will make [[Samba_CI_on_gitlab#Creating_a_merge_request|generating a merge request]] easier, as the branch name becomes the template title and allows ongoing distinct merge requests. |
|||
== Step by step instructions == |
== Step by step instructions == |
||
Line 45: | Line 55: | ||
Here are the basics steps for Samba team members (and others who have been granted access) to start from scratch. |
Here are the basics steps for Samba team members (and others who have been granted access) to start from scratch. |
||
Add remote: |
Add a git reference to the gitlab remote repository: |
||
<pre> |
<pre> |
||
$ git remote add gitlab-ci git@gitlab.com:samba-team/devel/samba.git |
$ git remote add gitlab-ci git@gitlab.com:samba-team/devel/samba.git |
||
</pre> |
</pre> |
||
Push current HEAD to '''new''' remote branch on gitlab CI: |
Push current HEAD to a '''new''' remote branch called '$USER-foo' on gitlab CI: |
||
<pre> |
<pre> |
||
Line 58: | Line 68: | ||
</pre> |
</pre> |
||
See [[Samba_CI_on_gitlab#Code_of_conduct|the Code of Conduct]] for an explanation of the usage of the username from the USER environment variable. |
See [[Samba_CI_on_gitlab#Code_of_conduct|the Code of Conduct]] for an explanation of the usage of the username from the USER environment variable. Note that if your gitlab username is different to the username on the local system, the '$USER' name you need to use in the git commands when pushing to gitlab is your '''gitlab''' username, not your local username. |
||
Push HEAD to existing remote branch on gitlab CI: |
Push HEAD to an existing remote branch called '$USER-foo' on gitlab CI: |
||
<pre> |
<pre> |
||
Line 69: | Line 79: | ||
CI results are here: [https://gitlab.com/samba-team/devel/samba/pipelines Pipelines] |
* CI results for changes are here: [https://gitlab.com/samba-team/devel/samba/pipelines Pipelines] |
||
* [https://gitlab.com/samba-team/samba/merge_requests|All merge requests] show a link to the Pipeline (CI) results for each patch series. |
|||
==Debugging CI failures== |
==[[Samba CI on gitlab/Debugging CI failures|Debugging CI failures]]== |
||
Please see these sub-pages for details on how to debug failures shows up by GitLab CI: |
|||
===In docker=== |
|||
* [[Samba CI on gitlab/Under the hood|How it works under the hood]] |
|||
Install docker and run: |
|||
* '''[[Samba CI on gitlab/Debugging CI failures|Debugging CI failures]]''' |
|||
==Where and how does CI run== |
|||
docker run -ti registry.gitlab.com/samba-team/samba:latest /bin/bash |
|||
We use a combination of [[Samba_CI_on_gitlab/Under_the_hood#Providing_the_private_VMs|Cloud VMs]] and [https://about.gitlab.com/2016/04/05/shared-runners/ GitLab.com shared runners] to provide the CI. |
|||
Then you need to clone the samba git repository with your changes and run the |
|||
test. |
|||
⚫ | |||
You can find how to run it in the log of the failed pipeline. |
|||
⚫ | |||
===make test=== |
|||
⚫ | |||
Many issues shown up in CI reproduce without difficulty by running the individual test. |
|||
==Free CI for forks on gitlab.com== |
|||
This can be either inside the docker container (which has the reference set of packages) or on your development system for more generic issues |
|||
By default a CI pipeline will run for any push to a [https://gitlab.com gitlab.com] fork of [https://gitlab.com/samba-team/samba Samba]. Much of the testsuite runs, but not everything, which is why we have the special shared development repository. |
|||
Build Samba with |
|||
⚫ | |||
./configure --enable-developer |
|||
make -j |
|||
===1 hour Timeout=== |
|||
And run the test with |
|||
You may see errors like this: |
|||
make test TESTS=mytest |
|||
ERROR: Job failed: execution took longer than 1h0m0s seconds |
|||
===Points to note=== |
|||
The script exceeded the maximum execution time set for the job |
|||
====Resource limitations==== |
|||
By default projects on gitlab.com have a '''1 hour timeout''' set on pipelines. This can be changed in the [https://docs.gitlab.com/ee/user/project/pipelines/settings.html project settings]. |
|||
The 'private' runners are 4 CPU virtual machines with 8GB of ram. These run in Rackspace's cloud and are paid for from a credit with RackSpace by the Samba Team. |
|||
We suggest using a timeout of '''3 hours''', which is still permitted on the free runners. |
|||
The 'shared' runners are 1 CPU virtual machines with 4GB of RAM. The name is a misnomer, they are not shared VMs, but access to the newly booted VMs is shared to us (and paid for) by gitlab.com. |
|||
=Creating a merge request= |
|||
Some tests fail or flap on GitLab CI due to resource limitations. This can cause |
|||
* If you are using your own private (forked) repository, then see our instructions on creating your [[Contribute#First_Merge_Request|first merge request]]. |
|||
* Docker failure code 137 (likely a kill -9 due to the out of memory killer running) |
|||
* Tests failure because they do not run fast enough (timeouts or failures due to timing) |
|||
* Race conditions (AD schema and DRS replication are particularly prone to this) |
|||
* If you pushed your patch to a '''unique''' well-named branch (above) in our shared development repo, then it is really easy to [[Contribute#Subsequent_Merge_Requests_.28and_complex_first_requests.29|submit the change for inclusion in Samba]]. |
|||
Tests should be re-worked to be more memory efficient, more robust to poor CPU scheduling and race-free, but in the meantime this is worth being aware of. |
|||
When you pushed your branch, a link will have been printed to create a merge request. Click that. If no link is printed it may be due to a complex ssh configuration issue. Don't panic, you can still create a merge request using the following section. |
|||
====Long hostnames==== |
|||
To do it later, navigate to your branch on gitlab, eg |
|||
''sn-devel'' is a nice short hostname, so is ''laptop'' etc. Specifically they are less than 14 characters, so do not need to be truncated. |
|||
[https://gitlab.com/samba-team/devel/samba/commits/npower-py2-py3-wip https://gitlab.com/samba-team/devel/samba/commits/$USER/foo] and click 'Create Merge Request'. |
|||
This will trigger an e-mail to Samba developers [https://gitlab.com/groups/samba-team/-/group_members registered on GitLab] suggesting that your code should be merged. You may wish to also mail the patch or '''merge request URL''' to '''samba-technical''' for additional attention. |
|||
Due to the way the GitLab CI instances are booted under docker, they get long hostnames like ''runner-191a8437-project-6378020-concurrent-0'', which sometimes cause difficult to diagnose issues if not always overridden in the test. |
|||
Further pushes to the same branch will then update the merge request, start a fresh CI run and re-send notifications, so the current code and test status is always available. |
|||
⚫ | |||
For this reason choose a '''unique branch name''' per patch series, and '''do not reuse''' the branch name until the merge request has been closed. |
|||
⚫ | |||
==Responding to feedback and updating the patch== |
|||
⚫ | |||
To update the patch in gitlab, simply re-push the branch with '''-f''' to force it to be overwritten: |
|||
git push gitlab-ci +HEAD:$USER/foo -f |
|||
=Merging patches from GitLab (for Samba Team members)= |
=Merging patches from GitLab (for Samba Team members)= |
||
Line 134: | Line 145: | ||
sadly doesn't work. |
sadly doesn't work. |
||
Then when the patch is in '''Samba.org's git master''' close the merge request a message like: |
Then when the patch is in '''Samba.org's git master''' close the merge request with a message like: |
||
Merged into master as <git hash> for Samba <next version>. |
Merged into master as <git hash> for Samba <next version>. |
||
Line 140: | Line 151: | ||
See for example [https://gitlab.com/samba-team/samba/merge_requests/9 this closed merge request]. |
See for example [https://gitlab.com/samba-team/samba/merge_requests/9 this closed merge request]. |
||
=Getting help= |
|||
⚫ | |||
If something isn't working, or you need help, the first port of call should be the [https://lists.samba.org/mailman/listinfo/samba-technical'''samba-technical''' mailing list]. |
|||
⚫ | |||
⚫ | |||
* The current status is at [https://status.gitlab.com/ GitLab status] |
|||
==[https://about.gitlab.com/support/ GitLab Support]== |
|||
* [https://forum.gitlab.com/ GitLab forum] |
|||
* [https://gitlab.com/gitlab-com/support-forum/issues GitLab.com Support Forum Tracker] |
|||
* [https://support.gitlab.com/hc/en-us/requests/new?ticket_form_id=334447 Contact GitLab for issues like account access problems] |
|||
==Don't panic== |
|||
Finally, in the immortal words of Douglas Adams: ''Don't Panic'' |
|||
As a toolchain, ''git'' is a decentralised version control system. While a service outage would certainly be inconvenient, development work can continue locally through an outage. |
|||
⚫ |
Revision as of 09:57, 15 September 2020
Samba has a GitLab mirror of the official git repository used to aid development
Where is Samba on GitLab?
- Samba's Official GibLab mirror is at https://gitlab.com/samba-team/samba
- The Samba shared CI repository is at https://gitlab.com/samba-team/devel/samba
Getting Access
Samba Team members
Samba Team members are encouraged to join the Samba Team organisation here:
Just ask on the team list from your team e-mail with your gitlab username.
A fellow team member with Maintainer rights can then add you as a Developer via this page:
Other Samba developers
Our broader developer community is encouraged to apply to use the Samba CI repo. Access is given to folks active on the mailing list with patches to submit, not just to Samba Team members.
Please write a mail to contributing@samba.org with your GitLab username, details of your contribution history or plans and a Samba Team member with Maintainer rights can then add you as a Developer via this page:
Triggering CI
The git remote URL for the Samba CI repo is
git@gitlab.com:samba-team/devel/samba.git
A CI run is triggered after every push. This makes it easy to find issues early.
Code of conduct
Please prefix branches with your gitlab username, and be nice. Use only to develop Samba. Don't overwrite the work of others.
$USER/topic
eg
frednurk/kerberos-timeout-handling
In return you get a full CI run using Samba Team provided resources running thanks to a credit in Rackspace's cloud. That in turn makes it easier for Samba Team members doing Code Review as your patches will work first time, and they can see proof of that.
If you describe your work in the branch name, this will make generating a merge request easier, as the branch name becomes the template title and allows ongoing distinct merge requests.
Step by step instructions
Here are the basics steps for Samba team members (and others who have been granted access) to start from scratch.
Add a git reference to the gitlab remote repository:
$ git remote add gitlab-ci git@gitlab.com:samba-team/devel/samba.git
Push current HEAD to a new remote branch called '$USER-foo' on gitlab CI:
$ echo $USER slow $ git push gitlab-ci +HEAD:refs/heads/$USER-foo
See the Code of Conduct for an explanation of the usage of the username from the USER environment variable. Note that if your gitlab username is different to the username on the local system, the '$USER' name you need to use in the git commands when pushing to gitlab is your gitlab username, not your local username.
Push HEAD to an existing remote branch called '$USER-foo' on gitlab CI:
$ git push gitlab-ci +HEAD:$USER-foo
CI Results
- CI results for changes are here: Pipelines
- merge requests show a link to the Pipeline (CI) results for each patch series.
Debugging CI failures
Please see these sub-pages for details on how to debug failures shows up by GitLab CI:
Where and how does CI run
We use a combination of Cloud VMs and GitLab.com shared runners to provide the CI.
CI for personal GitLab forks
Instructions how to run CI with your own GitLab runner can be found here: CI using Your own gitlab runner.
This is helpful if you do not wish to share the repository with others (you have your own fork on gitlab). However it requires your own build resources.
Free CI for forks on gitlab.com
By default a CI pipeline will run for any push to a gitlab.com fork of Samba. Much of the testsuite runs, but not everything, which is why we have the special shared development repository.
1 hour Timeout
You may see errors like this:
ERROR: Job failed: execution took longer than 1h0m0s seconds
The script exceeded the maximum execution time set for the job
By default projects on gitlab.com have a 1 hour timeout set on pipelines. This can be changed in the project settings.
We suggest using a timeout of 3 hours, which is still permitted on the free runners.
Creating a merge request
- If you are using your own private (forked) repository, then see our instructions on creating your first merge request.
- If you pushed your patch to a unique well-named branch (above) in our shared development repo, then it is really easy to submit the change for inclusion in Samba.
When you pushed your branch, a link will have been printed to create a merge request. Click that. If no link is printed it may be due to a complex ssh configuration issue. Don't panic, you can still create a merge request using the following section.
To do it later, navigate to your branch on gitlab, eg
https://gitlab.com/samba-team/devel/samba/commits/$USER/foo and click 'Create Merge Request'.
This will trigger an e-mail to Samba developers registered on GitLab suggesting that your code should be merged. You may wish to also mail the patch or merge request URL to samba-technical for additional attention.
Further pushes to the same branch will then update the merge request, start a fresh CI run and re-send notifications, so the current code and test status is always available.
For this reason choose a unique branch name per patch series, and do not reuse the branch name until the merge request has been closed.
Responding to feedback and updating the patch
To update the patch in gitlab, simply re-push the branch with -f to force it to be overwritten:
git push gitlab-ci +HEAD:$USER/foo -f
Merging patches from GitLab (for Samba Team members)
If the developer has created a merge request, then to merge, download the patch with (eg)
https://gitlab.com/samba-team/samba/merge_requests/12.patch
Add review tags and then 'git autobuild' locally. The merge button sadly doesn't work.
Then when the patch is in Samba.org's git master close the merge request with a message like:
Merged into master as <git hash> for Samba <next version>.
See for example this closed merge request.
Getting help
If something isn't working, or you need help, the first port of call should be the samba-technical mailing list.
gitlab.com Service Status
- gitlab.com service issues and outages are tracked on Twitter at GitLab Status
- The current status is at GitLab status
GitLab Support
- GitLab forum
- GitLab.com Support Forum Tracker
- Contact GitLab for issues like account access problems
Don't panic
Finally, in the immortal words of Douglas Adams: Don't Panic
As a toolchain, git is a decentralised version control system. While a service outage would certainly be inconvenient, development work can continue locally through an outage.