Writing programs that drive Jenkins

One of the interesting discussions during SCALE 10x was about using Jenkins as a piece of a bigger software. This person was interested in using Jenkins to run some business analysis operations, and wanted to have a separate interface for business oriented people. This is actually an emerging but common theme I hear from many users. Another company in San Jose actually built the workflow engine that uses Jenkins as a piece in a bigger application (aside from the actual build and test, this workflow involves reviews, approvals, etc.), and GitHub Janky can be classified as one such app, too.

This is something I always believed in — that every piece of software needs to be usable by a layer above. Or put another way, every software should be usable as a library.

So in this post I’m going to discuss various ways you can programmatically drive Jenkins, which is ideal for editor jobs on top of everything else.

Let’s start with the REST API of Jenkins. For most of the data Jenkins renders as HTML, you can access its XML version and JSON version (as well as a few other formats, like Python literal fragment.) You do this by adding /api to the page (see http://ci.jenkins-ci.org/api for example.) Those pages discusss other REST API where applicable. For example, you can POST to certain URL and it’ll create/update job definitions, etc.

If you are going to use REST API, you might find the auto-discovery for Jenkins useful. You can discover Jenkins on the local subnet via UDP broadcast, or DNS multi-cast. There’s also a distinctive HTTP header “Jenkins-Version” on the top page of Jenkins that allows your application to verify that it’s talking to a real Jenkins server, as well as an instance ID that allows you to identify Jenkins instanecs. These features allow you to build smarter applications.

For Jenkins protected by some authentication mechanism, you can use the user name + API key in the HTTP basic auth (and I want to add OAuth support here.)

REST API is great that it’s programming language agnostic. It is also convenient that neither the server nor the client has to trust each other. But those APIs are bound by the request/response oriented nature of the HTTP protocol.

Another great integration point for Jenkins is the CLI. This uses the same underlying technology that drives master/slave architecture, which enables your command line clients to be a lot more intelligent. For example, REST API exposes an URL that you can post to get a build started. But the equivalent command in CLI can have you block until the build is complete (and exit code indicates the status), or run the polling first and proceed to build only when the polling detects a change, or allow you perform a parameterized build with multiple file uploads very easily. For protected Jenkins, CLI supports SSH public key authentication to securely authenticate the client.

A slightly different version of the CLI is “Jenkins as SSH server”. Jenkins speaks the server side of the SSH protocol, and allow regular SSH clients to execute a subset of CLI commands. In this way, you don’t need any Java runtime installed on the client side to drive Jenkins.

These two integration APIs are often much easier to script than REST API.

Those APIs are available for non-privileged users, and they are great for small scale integrations. But for more sophisticated integration needs, we have additional APIs.

One is the REST API access to the Groovy console, which allows administrator users to run arbitrary Groovy script inside the Jenkins master JVM (and you can submit this script as POST payload, and get the response back as the HTTP response.) This allows you to tap into all the Jenkins object models. Unlike the REST API, in this way you can ship the computation, so in one round-trip you can do a lot. You can do the same with CLI, which also lets you access stdin/stdout of the CLI.

The other sophisticated integration API I wanted to talk about is the remoting API that does Java RPC (not to be confused with the remote API, which is the synonym for the REST API.) The remoting API is the underlying protocol that we use for master/slave communications, and it revolves around the notion of shipping a closure (and code associated with it) from one JVM to another, execute it, and get the result back. If your application runs elsewhere, you can establish the remoting API channel with Jenkins master, then prepare a Callable object. You can then have Jenkins master execute this closure, and the result is sent back to your JVM.

There’s an example of this available. You bootstrap this in the same way the CLI client talks to the master, then you “upgrade” the communication channel by activating the remote code download support (which requires the administrator privilege, for obvious reasons.)

The great thing about this is that your data structure is rich Java object model all the way, and you never have to translate your data to externalizable serialization data format like XML or JSON. This greatly simplifies your program.

I think this list covers all the major integration APIs that Jenkins offers. If you are building any interesting applications that uses Jenkins as a building block, please share your experience so that we can make it better!

17 Comments Add yours

  1. Hey man,

    Thanks for this sharing. It is really nice stuff. Can you tell me if the EC2 plugin for Jenkins is available via REST ? I guess it might be as it just is extension to Jenkins but wanted to know your thoughts about it.

    Also if you can provide some sample code which shows interaction specifically with the EC2 plugin that be great.

    Thanks agin,

  2. Great post! Am using Jenkins to build bioinformatics tools 🙂 http://www.biouno.org

    At moment we are creating only a set of plug-ins for specific tools, but in some near future we will need to work on UI and external integration in Jenkins.

    Thanks for all this great info.

    All the best

  3. Ghassen Ben Mansour says:

    Hi Kohsuke ,
    thanks for sharing.I want to experiment the remoting api from the cli-channel-demo. but I have the following exception :

    Exception in thread “main” java.io.FileNotFoundException: http://localhost:8080/cli
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
    at hudson.cli.FullDuplexHttpStream.(FullDuplexHttpStream.java:77)
    at hudson.cli.CLI.connectViaHttp(CLI.java:150)
    at hudson.cli.CLI.(CLI.java:134)
    at hudson.cli.CLI.(CLI.java:107)
    at hudson.cli.CLI.(CLI.java:99)
    at hudson.cli.CLI.(CLI.java:91)
    at org.jenkinsci.demo.cli.App.main(App.java:27)

    Can you help me with that ? another question is it possible to experiment the remoting api without using jenkins server ?


  4. kohsuke says:

    The error seems to indicate that you don’t have Jenkins running on localhost port 8080.

  5. rick says:

    I’m trying to run the cli-channel-demo for the removing api but cannot because some of the dependencies are not available. Do I need to extract these jars out from the war file? Or are they available in a maven repo?


  6. rick says:

    Are there any good samples for pragmatically controlling jenkins using the remoting api? I know that I can go thru the javadocs of jenkins and try to figure out what everything does but was hoping for some good snippets somewhere to show me some of the common things that can be done.

  7. Skúli says:

    Fantastic write up! I’ve been working on hubot (http://hubot.github.com/) scripts for our HipChat bot, and integrating them with Jenkins has been a priority for me (not just triggering builds, but gathering all sorts of information). The REST api works like a charm. Kudos!

  8. kohsuke says:

    @Deependra Shekhawat
    Plugins are responsible for exposing their functionalities via REST API/CLI. Exposing is easy, but it’s just so easy for plugin developers to forget to expose them.

    So if you have specific functionalities in EC2 plugin that needs to be exposed, it’d be great if you can file tickets.

  9. kohsuke says:

    I updated the POM to define <repository> entry that helps Maven finds the right artifacts.

  10. kohsuke says:

    CLI channel demo was meant to be the example of the remoting API. Or do you mean remote API (which is a synonym for the REST API)?

  11. kohsuke says:

    A good example of using the REST API is Nestor, a node.js command-line tool built on top of the REST API.

  12. Nice example. But I have a problem with the API:

    In a mavenmoduleset project, how can I retrieve the main module?
    I know there is module entries, but there is not order that guaranties that first or last element is the main module.
    Understand the main module as the name of the groupId:artifactId of the top level pom.xml.

  13. Ron Pomeroy says:

    Is there a particular reason the install-tool command of the jenkins-cli cannot be executed from outside a job? I’d like to script the setup of Jenkins (without having to resort to heavyweight tools like Chef/Puppet) to make it repeatable and to be able to store the setup code in scm


  14. subhasis says:


    Can you refer me something or some companies who have already developed this features. I am looking for something like this where I want to build my own UI on top of Jenkin for that matter all CI tools. Through the UI any one can perform build, provision infrastructure, deploy, monitor and maintenance.


  15. Milo says:

    Hi Subhasis, were you able to develop UI to invoke Jenkins? We are trying to achieve same. Could you share details about how we can invoke Jenkins throught UI code?


  16. amit says:

    I’m looking for samples to start building UI plugins in jenkins using groovy.

  17. Chitra says:


    Can you please guide me if I can do REST API Post call with input request parameters structured in xml way from jenkins?



Leave a Reply

Your email address will not be published. Required fields are marked *