Polling must die: triggering Jenkins builds from a git hook
As I keep saying, polling a repository from Jenkins is inefficient; it adds delay on the order of minutes before a build starts after a commit is pushed, and it adds additional loads. It is much better instead to do push-notification from the repository. In this post, I’m going to explain how to do this for Git, which brings this on par with Subversion.
The previous best practice of doing this is best summarized in this blog post. While this works, this is less than ideal. A part of the problem is that this requires hard coding of job names inside the repository hooks, making it hard to keep them up-to-date. Another problem is that if your job only cares about one branch in a busy repository, you don’t want a new build to be triggered. Finally, the last problem is that you need some extra work for secured Jenkins.
With the latest Git plugin 1.1.14 (that I just release now), you can now do this more easily by simply executing the following command:
curl http://yourserver/jenkins/git/notifyCommit?url=<URL of the Git repository>
This will scan all the jobs that’s configured to check out the specified URL, and if they are also configured with polling, it’ll immediately trigger the polling (and if that finds a change worth a build, a build will be triggered in turn.) This allows a script to remain the same when jobs come and go in Jenkins. Or if you have multiple repositories under a single repository host application (such as Gitosis), you can share a single post-receive hook script with all the repositories. Finally, this URL doesn’t require authentication even for secured Jenkins, because the server doesn’t directly use anything that the client is sending. It runs polling to verify that there is a change, before it actually starts a build.
One more final note — in Git, unlike Subversion, a repository does not have its own identity, and a single repository sometimes have multiple URLs to access it. In such a case, simply execute the curl commands multiple times with all the different URLs.
That’s it. I hope this will help reduce the use of polling and have more people switch to push notifications. It really is addictive to see the build start in a split second after you push a change.
@Jay This extension was only made to the Jenkins code base, so it won’t work with Hudson. Switch to Jenkins for the latest and most active continuous integration developments.
Can this work with GitHub as well ? Or do I have to use the GitHub Plugin
to do something similar if I have GitHub Enterprise internally at work ?
@mns
No, GitHub doesn’t allow you to directly touch post-receive hook. You should use the GitHub plugin for it, and it should work with GitHub Enterprise, too.
@eedri
I doubt if this works with multi SCM plugin. “Multi SCM” feature is something we really need to bring back into the core.
Unfortunately, I can verify this does not work with the MultiSCM plugin.
I’ve read there were plans to rewrite the git plugin to natively support multiple git repos, but given the last comment date was quite awhile ago, I’m wondering if this is happening or not. Or maybe I’m just out-of-the-loop here.
I am having trouble executing the build with this trigger in timely manner.
Namely, the trigger is executed, and the job is also found (and is valid / scheduled) but the build occurs about 10-20 minutes later.
I’ve done a bit of investigating and this is the flow and the information I can gather from logs:
* the project (e.g. jenkins-hooks-test) has a git(lab) repository, with the following web hook: http://ci-server/jenkins/git/notifyCommit?url=http://git.infobip.local/jenkins-hooks-test
* the project has received a push to the repository
* I can see that the hook fired the notifyCommit on jenkins, this is the trigger log:
01.10.2012. 14:46:46 hudson.triggers.SCMTrigger
FINE: Scheduling a polling for hudson.maven.MavenModuleSet@115be90[jenkins-hooks-test]
* the build starts 7 minutes later, the polling log:
Started on 01.10.2012. 14:53:00
Using strategy: Default
[poll] Last Build : #22
[poll] Last Built Revision: Revision 74a83490028ebf3b15af36a90d443f98a5e8262c (origin/master)
Fetching changes from the remote Git repositories
Fetching upstream changes from git@git.infobip.local:jenkins-hooks-test.git
Polling for changes in
Done. Took 32 sec
Changes found
So, from the chain of events point of view, this all OK since a change in the repository fires the build/polling trigger. The only problem I have is the delay. I’ve tested this when Jenkins is loaded with jobs but also when it’s building nothing.
The project is configured to Poll SCM @yearly and Build whenever a SNAPSHOT dependency is built. No other build triggers are present.
The main Jenkins configuration has the Quiet period set to 0.
I am using Jenkins 1.483 and git plugin 1.1.24.
Jenkins is configured with about 260 projects, of which most is currently using scheduled polling (every minute usually). I counted these by viewing all projects on the dashboard since the following snippet returns 1252 projects:
println Hudson.getInstance().getAllItems(AbstractProject.class).size()
(can’t figure out why the numbers don’t match)
I’ve also tried executing the trigger with this url: git@git.infobip.local:jenkins-hooks-test.git, and I get the same behaviour.
If used from curl, the response is:
“Scheduled polling of jenkins-hooks-test”
which notes that the build/polling trigger performed OK
(from what I can see in plugin code).
The only thing that is suspicious is probably the isStarving() on the trigger.
I’ve used the below script to get the information and it is in fact starving for 5 minutes (even more). I’ve also used the same script to run the trigger manually, with the same effect.
import hudson.plugins.git.*
import hudson.triggers.*
for (AbstractProject project : Hudson.getInstance().getAllItems(AbstractProject.class)) {
if(project.name.equals(“jenkins-hooks-test”)) {
println “project ${project.name}”
SCMTrigger trigger = project.getTrigger(SCMTrigger.class)
//println “executing the build trigger on the project”
//trigger.run()
println “the trigger is starving: ${trigger.descriptor.queue.isStarving(5 * 60 * 1000)}”
}
}
Any help appreciated!
P.S. If there is a smarter way to use gitlab with this plugin, I’d be very happy to give a hand…
P.P.S. the following starts the build instantly…
Maybe this is OK, since we are in fact “notifying of change to be built” ?
import hudson.plugins.git.*
import hudson.triggers.*
import hudson.model.*
for (AbstractProject project : Hudson.getInstance().getAllItems(AbstractProject.class)) {
if(project.name.equals(“jenkins-hooks-test”)) {
println “building project ${project.name}”
project.scheduleBuild(new Cause.RemoteCause(“gitlab”, “changes in the build”))
}
}
@Vanja Radovanović
I’ve created a gitlab hook plugin that offers both notify commit and build now options. This is not a direct solution but it works for us. You can find details @ https://wiki.jenkins-ci.org/display/JENKINS/Gitlab+Hook+Plugin
Hi. Can You tell me, please, where I need to add this command in jenkins?
How do I get this to work with all users of the repository? Git hooks appear to be user specific and are not saved in the repository.
I could not get the correct refs, but this worked for me:
for ref in $@; do
IFS=’/’ read -ra REF <<< "${ref}"
branch="${REF[2]}"
branch_list=${branch_list}${delim}${branch}
delim=","
done
I think its because we still have a old git version perhaps.
I have the setup working for git branches. Any new change triggers the builds configured.
@kohsuke
How do I make it work for any new git tag created manually? (As soon as a new tag is created, the hook causes the builds configured to trigger).