Jenkins ChromeDriver plugin

I released Jenkins ChromeDriver plugin that auto-installs chromedriver on all slaves. Here is why this is useful.

ChromeDriver is needed to run WebDriver-based tests with Chrome. Unfortunately this is a native program, so in a heterogenous Jenkins, you need to make sure you have the right binary available on each of the slaves. While that’s not hard, it’s awfully tedious. This plugin fixes that by automating it.

This plugin would be a nice companion to the recently upgraded Selenium Grid plugin that automatically runs Selenium Grid on top of your Jenkins slaves. When you combined, you just install two plugins to the Jenkins master, and without touching slaves at all, you got the whole cluster instantly ready for Selenium2 WebDriver testing. How is that?

Jenkins now acts as an SSH daemon

Starting in the upcoming Jenkins 1.446, Jenkins speaks the server-side of the SSH protocol. I think this is exciting, and here is why.

For quite some time now, there is Jenkins CLI, which lets you access Jenkins from command line by using a custom client jar file. There are several dozen commands available (that you can check by visiting http://server/jenkins/cli), ranging from creating a job to shutting down Jenkins. These CLIs are often a lot more scriptable than HTTP interfaces, and so it is often used by other programs to interact with Jenkins.

Jenkins 1.446 exposes a large subset of these commands through SSH, making it even easier to access this functionality. You no longer need any special client jar nor Java VM to talk to Jenkins. All you need is a stock ssh client. As I explained earlier, Jenkins lets you register your SSH public keys, and that is how you authenticate.

As always, the server side of this is extensible. Plugins can define additional commands to be exposed via SSH, which is a great building block for more sophisticated integrations. This is particularly so because SSH is a standard transport protocol for many tools, such as Git, Subversion, and so on. I’m planning to write a few plugins that take advantage of this in the coming days.

Wiki article talks more about the details, including how you can discover the endpoint, how you can verify the server key, and so on.

(And no, this is not about making Jenkins a general-purpose SSH server that you can use to login to the server.)

Upcoming San Francisco training

CloudBees will be hosting another training in San Francisco Bay Area in January 26, 2012, and Feburary 23rd in Tokyo.

This 1-day training starts with the basics and then covers some of the advanced techniques, especially in combination with some plugins. As we deliver this training more, we gradually adjusted the material to cover more advanced topics, like promotion, build pipeline, parameterized triggers, and so on — the kind that’s getting more traction lately as Jenkins starts to encompass continuous delivery.

And for me, this training will be the first to use the latest “install plugins without restart” feature, so that should help with attendees keeping the focus a bit.

In these trainings, we cap the number of attendees to a fairly small size so that I can pay attention to each attendees enough. People normally bring in questions that go beyond the training curriculam to discuss the problems they face in their day-to-day Jenkins administrations. I welcome those, too!

If you are interested, please sign up while the seats are available!

GitHub releases Janky

I saw an announcement that GitHub released Janky today.

I initially got confused a bit because the post says Janky is a continuous integration server, which got me thinking that it’s something you’d use instead of Jenkins, but as I read more about it, it became clear that it’s something you use in conjunction with Jenkins — it’s an application that sits between GitHub API, Jenkins API, and Hubot chat bot to help you create jobs on Jenkins, set up build triggers from push notifications, and so on. So for those who are using GitHub for hosting repositories and Campfire for chat, this is a nice interface that lets you get more out of your Jenkins.

The other thing I like here is that Janky pretty much requires you to run all the services “on the cloud”, because they need to talk to each other via HTTP calls. This makes it a great fit with CloudBees’s DEV@cloud, the hosted Jenkins service CloudBees provide. So between GitHub as a hosted service for repositories, Campfire for a hosted service for chats, and DEV@cloud for a hosted service for Jenkins, you can start to see the power of integration between hosted services. Greater collaboration of web services is a future I can believe in!

And if you like the idea of GitHub + Jenkins but you don’t do Campfire, you might find the Jenkins GitHub plugin useful. This plugin lets you set up a push-based build triggering from your GitHub project with a single checkbox.

Finally, as an icing on the cake, they got this nice thing to say about Jenkins:

The power, vast amount of plugins and large community of the popular CI server all wrapped up in a great experience.

One more validation to the great momentum of the Jenkins project. Yay!

Congrats to the GitHub guys for the launch of Janky!

Webinar: Jenkins Enterprise by CloudBees

On Janurary 10th, I’ll be doing a webinar about Jenkins Enterprise by CloudBees (formerly known as Nectar, which we renamed with the permission of the community.)

Jenkins Enterprise is Jenkins LTS + a number of CloudBees’ value-add plugins that help large and serious users, with the support. This release contains a number of new plugins, most notably the template plugin, which allows people to manage a large number of similar jobs, creating more domain friendly configuration mechanism, as well as means for the administrators to force certain practice on jobs.

The release also includes improvements to existing plugins — folders can now have views defined within them, and with VMWare, you can now tell Jenkins to use all computers in a specific folder, as opposed to configure individual machines one by one. There’s also a new job scheduling mode that makes Jenkins prefer idle slaves over partially used slaves that already have your workspace. And so on.

This release also lets you run these value-add plugins on top of non-LTS stock Jenkins.

All things combined, I think we got something interesting for many Jenkins users. Hope you can join us in this webinar.

Jenkins Plugin Tip: Access control and visibility in actions

As I discussed before, Action is one of the primary ways plugins use to add more information to the top page, project pages, build pages, and so on.

Today I’d like to talk about writing protected actions that require some kind of access control. Depending on your requirements, there are several ways to do this. There are three parts in Action that affects behaviours with this regard.

One is the getIconFileName() method. As its javadoc states, returning null from this method hides your action from the HTML page. So if you have an action and you don’t want to show it for users who don’t have the permission to access it, it could be something like the following:

public String getIconFileName() {
    return Jenkins.getInstance().hasPermission(Model.LIST)
         ? "gear.png" : null;

The next is the getUrlName() method. This method also allows null as the return value. This might sound similar to getIconFileName() but the implication is different. With getIconFileName() returning null, the rendered HTML page won’t show the link but if someone knows the URL, they can still access it. But with this method returning null, it’s as if no such URL is recognized.

So with the code like the following, if someone without the permission requests it, he’ll get 404 not found (modulo a bug, which I just fixed toward 1.443). This is sometimes desirable, as not only can you hide the data, but you can also hide the fact that something exists. For jobs, Jenkins does this — http://jenkins/job/top-secret-project/ returns 404 not found, not 401 forbidden, so guessing the project name will not help attackers gain any information.

public String getUrlName() {
    return Jenkins.getInstance().hasPermission(Model.LIST)
         ? "template" : null;

But this is sometimes undesirable, as users will not be prompted for authentication. If someone hits the link with an expired session, he’ll get 404 not found and he needs to be smart enough to know that this is because he hasn’t logged in, then navigate manually to the login page and come back. To avoid this problem, you’ll do the following and advertise the URL all the time.

public String getUrlName() {
    return "template";

And that brings me to the third part, because getUrlName() now always returning non-null means you aren’t actually checking if those who are accessing has a permission to do so. To do this, you use StaplerProxy.

class MyAction implements Action, StaplerProxy {
    public Object getTarget() {
        return this;

What happens is that when the URL that someone requests hits this action (or something underneath), we’ll verify that the requestor has the permission. If not, this will initiate the authentication, such as redirecting the user to the login page, or initiating the OpenID protocol with the preconfigured identity provider. Normally this interface is used to defer the UI processing to another object (sort of like how symlink works), but in this case we return this to indicate that we process this request by ourselves, and it’s smart enough not to cause infinitely recursion.

I guess the rule of thumb is that (1) you have to check the permission either in getUrlName() or getTarget(), or else there’s no access control, (2) you use getIconFileName() control the visibility in the HTML page, and (3) you use getUrlName() to control if you want 401 or 404 in case the access is denied.

Installing plugins to Jenkins without restart

Today I’m going to talk about the upcoming feature in Jenkins 1.442 (to be released today), which allows you to finally install plugins without restarting Jenkins.

As I’ll discuss later, internally this is somewhat of an involving work. But for users, this is a very simple feature that doesn’t involve any learning curve. You’ll go to the plugin manager, click “Available” tab, then choose the plugins to install. Scroll all the way down, then you’ll see the “Install without restart” button as well as the “Download new and install after restart” button. The former is the result of this work, allowing you to start using the new plugins right away. The latter is the traditional behaviour, where new plugins take effect after the next restart.

I’m even thinking about removing the latter button after a while, once we gain enough confidence in the feature. I can’t think of any inherent reasons the latter is desirable over the former.

Click the button on the left, and the plugin gets downloaded, installed, and activated:

Whereas existing users would by now be “trained” to click the “restart Jenkins when done” checkbox, now you can just go back to the top page and start using the new features from this plugin right away. Yup, that’s all there is to it.

But how about upgrades?

Unfortunately, because of the architectual choice made in Jenkins, this same scheme wouldn’t allow us to upgrade existing plugins. See below.

But how about uninstallation?

Likewise, this same scheme wouldn’t really let us do the “uninstallation” in the normal sense of the word. But I think some fake of that might be possible — like disabling all the contributions from this plugin (which will prevent anyone from using builders/publishers/etc in newly created jobs), but leaving existing jobs with those features configured as-is, until the next reboot, at which point those features will be just forgotten.


Jenkins added the plugin support in 1.44, which is some 400 releases ago (man I’ve been doing this for long time!), and since that time Jenkins required a restart for new plugins to take effect. This is because we load plugins at start-up, doing all kinds of computation to build up the immutable data structure around all the plugins. Thus requiring a restart was partly out of my laziness, partly my preference to the immutable data structures, and partly because the computation was rather sequential. And I had bigger fishes to fry, like update centers, and things were left as it was.

Then later came the improvement to the boot up, which removed a long sequential code, and divided them up to the directed acyclic graph of smaller tasks, which we then execute in parallel. This was primarily to speed up the boot time, but this made it somewhat easier to do dynamic initialization as a side effect. Then further down the road, the introduction of Guice made it harder once again, as it prefers immutability, too. So the current implementation puts the new components into the child injector, which seems to make enough plugins happy.

Finally, why is upgrade hard? This is because Jenkins keeps instances of model objects for long time in memory (unlike, say, a typical database application, where those things are request scoped and thus much short-lived.) I like this for a number of reasons, such as more straight-forward object traversals, and the fact that builds take a long time anyway. But as with everything else in the software design, it is not without downsides, and one of them is that it makes dynamic reloading harder. This hopefully answers why just using OSGi alone does not fix this problem.

My current thinking is that “fixing” this via HA is probably the way to go. Or maybe fake uninstallation + dynamic installation might be enough for most plugins. We’ll see.


Hopefully you’ll like this feature. While this is an obvious simple feature for users, as you see internally I’ve done some hard work. If this makes it easier for you to try out new plugins, my time was well spent.

Polling must die: triggering Jenkins builds from a git hook

As I keep saying, polling a repository from Jenkins is inefficient; it adds delay on the order of minutes before a build starts after a commit is pushed, and it adds additional loads. It is much better instead to do push-notification from the repository. In this post, I’m going to explain how to do this for Git, which brings this on par with Subversion.

The previous best practice of doing this is best summarized in this blog post. While this works, this is less than ideal. A part of the problem is that this requires hard coding of job names inside the repository hooks, making it hard to keep them up-to-date. Another problem is that if your job only cares about one branch in a busy repository, you don’t want a new build to be triggered. Finally, the last problem is that you need some extra work for secured Jenkins.

With the latest Git plugin 1.1.14 (that I just release now), you can now do this more easily by simply executing the following command:

curl http://yourserver/jenkins/git/notifyCommit?url=<URL of the Git repository>

This will scan all the jobs that’s configured to check out the specified URL, and if they are also configured with polling, it’ll immediately trigger the polling (and if that finds a change worth a build, a build will be triggered in turn.) This allows a script to remain the same when jobs come and go in Jenkins. Or if you have multiple repositories under a single repository host application (such as Gitosis), you can share a single post-receive hook script with all the repositories. Finally, this URL doesn’t require authentication even for secured Jenkins, because the server doesn’t directly use anything that the client is sending. It runs polling to verify that there is a change, before it actually starts a build.

One more final note — in Git, unlike Subversion, a repository does not have its own identity, and a single repository sometimes have multiple URLs to access it. In such a case, simply execute the curl commands multiple times with all the different URLs.

That’s it. I hope this will help reduce the use of polling and have more people switch to push notifications. It really is addictive to see the build start in a split second after you push a change.