Archive

Archive for the ‘jenkins’ Category

Jenkins commit activities in 2012

January 25th, 2013

Stephen is having fun with Git and R, and I saw another person creating a similar comparison (although I’m not sure where the credit should go on that one.)

Both charts are great, but I also noticed that their Y-axis aren’t consistent. So if people aren’t careful they might not see Jenkins as favorably as it actually is. So here is the graph, comparing # of commits in the year 2012. They both only includes the core and in the master branch:

On average, we have 41.2 commits per week, compared to 9.7 on the other side. If you want the raw data, here is the Excel file.

hudson, jenkins

On the road for the rest of the month

January 10th, 2013

I’m really excited to kick off my 2013 with a tour around the world for Jenkins.

The first stop will be in Tel Aviv, where I’ll be doing training with AlphaCSP. This has sold out, but AlphaCSP will be delivering this training again in the future. I’ll then head to London for the Jenkins User Event in London, with James Nord (of the m2 release plugin fame) and Andrew Phillips from XebiaLabs. This one has also sold out, but I’ll be recording the talks and posting them online. After tha, I’m in Munich to deliver another Jenkins training with ObjectBay. Again the same deal here — it’s sold out but they’ll be scheduling repeats later.

I’ll then be back to my home for a few days, then off to Seoul for the first-ever Jenkins meet-up in Korea. I’m really excited about this, and I’m trying to kick-start a local community, which is invaluable in this region of the world. If you are in the area, please come join us, and get to know other people in the Jenkins community.

Then I head to Tokyo to an user meet-up (no registration page is up yet, please check back our website later), and a couple more trainings (this one has more seats available.)

From there I’m flying to Brussels for FOSDEM. They gave me a keynote slot (yay!), and Tyler is organizing the testing and automation devroom. A number of Jenkins people will be there, and we’ll have a table for two days. This will be exciting!

And in my last leg of the journey, I’ll be at JFokus in Stockholm. They kindly gave me a long 3 and half hour slot to talk about Jenkins in depth, aside from more typical 50 mins talking slot. I’m looking forward to this new format.

And when I finally get back to home, I suspect I need some down time to recover.

If you are in the area nearby, and grab some stickers or want to say hello, please let me know! And my apologies in advance for the delay in responding e-mails and other disruptions.

jenkins , , , ,

The other side of forking and pull requests

January 4th, 2013

Charles Nutter of JRuby fame had this tweet yesterday:

And this touches on something I’ve been thinking for a long time.

My experience mostly comes from the Jenkins project, which is one of the open commit policy projects. Everyone gets to be a committer just by asking, and that grants them access to all the plugins. This policy is deep in our culture that we even have an IRC bot that the project people use to add new committers.

This is not necessarily a good idea for every project, but it certainly served us well, especially back in the days when we had the entire source code in the Subversion repository. During that time, for people to make changes to the plugins, they really needed to be a committer — maintaining a custom patch set on top of an external Subversion repository is a royal pain in the back side. The open commit policy means anyone interested in committing can join the project without a fuss, and once they become a contributor and start commiting code, they feel like being a part of the project, and that feeds fresh blood into the Jenkins developer community.

While I totally believe in the technological superiority of Git as a version control system over Subversion, in my view, the fork and pull-request driven world of Git hasn’t been a complete win for the Jenkins project.

Forking and pull requests do encourage contributions by eliminating certain barriers to start hacking on someone else’s project. Whereas it was quite painful to maintin patches externally to someone else’s source code in Subversion, Git makes this trivial with branching and occasional rebases or merges. This part is a good thing.

On the other hand, this ease of maintaining your own versions reduces the incentive to bring your changes back to the community, to some extent. After all, you’ve already scratched your itch and it works on your machine. Why bother trying to push the code back and go through the trouble of writing tests and documenting the change, convincing others about the design, and arguing why that change was needed to begin with. You can see this in a number of forks people created that never turns into as pull requests, and I’ve done this a number of times myself.

An even bigger problem in my mind is that by allowing people to contribute by just sending pull requests, it encourages the “throw the stuff over the wall” way of contributing. Sometimes people can do stuff without telling us anything upfront, and you as a receiver would be hard-pressed to turn it down, given that someone spent so much effort in it. Sometimes when we request additional changes, they never come back. It makes the collaboration and communication after the fact, not before the coding. And it also makes the pathway to the committer bit unclear. How do we convert people from the side sending pull requests to the side receiving pull requests? If we have to ask them one by one it won’t scale.

The situation gets worse on plugins, where there may not be an active maintainer at the moment. We’d like those who are sending us pull requests to take over the maintainership, and use their best judgement to evaluate the changes. But when GitHub makes them feel like the proper way to contribute is by sending a pull request, and if your pull requests do not get any attention, I’m not sure how they are supposed to realize that the plugin needs a new owner. Back in the days of Subversion, the repository alone was enough to be a center of gravity of the project. It attracts new developers (so long as we let them be, aka open commit policy) and it’s almost self sustaining. Now it takes more than just the repository.

In any case, I just wanted to point out that the impliations of forking and pull requests are bit more nuanced.

P.S. I’ve been long meaning to write a bot that patrols for such unattended pull requests, and encourage them to become committers, but haven’t had a chance to do so yet. I still intend to do this, and we’ll see how it goes.

jenkins , , ,

In São Paulo from this weekend

November 26th, 2012

I’ll be flying to Sao Paulo this week to attend Jenkins meet-up (Saturday) and JavaOne Latin America (next Tuesday and onward).

The last time I visited Brazil was a few years ago, but thanks to Mauricio Leal, whom I tagged around with for a JUG tour around Brazil, it was really a blast. This time around I don’t have the benefit of the local guy to the degree I had before, but I’m really looking forward to it.

The major pre-travel issue was that the Brazilian government run out of the visa stickers. But I made it just in time (or more precisely I hopefully will be — I’ll be picking up my passport Wednesday and flying Thursday. Phew!)

The highlight of the trip will be the first-eve Jenkins meet up in São Paulo — I’ve been a big fan of local communities where there exists a language barrier. I’m from Japan, so I’ve seen it the first hand the value of local communities would bring to the table. The speaker/session line up is amazing, the event is free, and there’ll be a social at the end, so if you are in the area and has been interested in meeting with fellow Jenkins developers and users, please RSVP.

But if you can’t come in person, at least please consider joinining the Brazilian Portaguese mailing list of Jenkins users, which we are launching.

I don’t really have any plans on Sunday and Monday — any sight-seeing suggestions and/or interest of meeting up and chatting about stuff would be more than welcome. Leave the comment here, or see the right for my e-mail address.

jenkins , , ,

POTD: submit a patch to Jenkins, and let him test it for you

October 8th, 2012

Here’s my 2nd after-JavaOne “project of the day” Jenkins plugin. This has been in the back of my mind for quite some time, but it took this gentleman to grill me on this feature during JavaOne for me to finally put it together — so thank YOU for doing that. although I didn’t catch your name.

The plugin is called the “patch parameter” plugin. This plugin lets you submit a patch when scheduling a build, and that gets applied to the checked out source tree before a build would commence. Any failure to apply a patch will result in a build failure.

This plugin can be used for a “pre-tested commit” workflow. You can work on a change locally, have the diff tested on the server, then if you are satisfied, you can commit it. I can imagine this would be also an interesting building block for integration with code review tools.

I should also note that we have the Subversion Merge plugin for a different approach to a similar problem, and distributed VCS can generally do this better (for example I do one for Git in CloudBees Validated Merge plugin.)

jenkins, potd , ,

POTD: iOS device connector plugin (cont’d)

October 7th, 2012

Today, I wrapped up the project I started Friday and released iOS device connector plugin.

In addition to listing all iOS devices, this plugin lets you deploy IPA files from anywhere (in Jenkins build via a build step or outside via CLI) to any of the connected iOS devices.

There are still some loose-ends that I’d like to get some feedback on:

  • You should be able to select a device without electing a specific UDID, and I’m curious what kind of criteria people needs here. Do you need to be able to say “deploy my app on iPad2 with 16GB memory”, or is “deploy my app to an available iPad” suffice?
  • For this to be useful for automated test execution. I think we need to be able to launch the app (and I assume the tests embedded in the app itself will run and shuts itself down in the end?) How do people do it?
  • Don’t we need to inject the coordinates of the actual device the app was deployed to, so that your build can do more stuff with it?
  • What are other interesting operations to devices. File access? Screenshot?

Anyway, the plugin is already released. I hope you find it useful.

jenkins, potd

POTD: iOS device detection in Jenkins

October 5th, 2012

I was talking to my colleague Mark Prichard about mobile development with Jenkins, and I came up with this idea.

If you are doing iOS app testing with real devices, you need to tether the device with a computer so that you can push the app-to-be-tested to the device. In a local development environment, you’d naturally do this by hooking up a device to your laptop.

But in the CI environment, we can do better. Imagine if you have a Jenkins master connected to a dozen Mac minis, where each Mac mini is then connected to various iOS devices. There’s nothing stopping Jenkins from listing up all the devices connected to all the slaves. Then based on this information, Jenkins can lease devices to builds.

Say someone wants to run a test somewhere. Jenkins can find an unused device and push it, then inject the coordinate of that device to the build. Or how about the axis in a matrix project that allows you to say “I need this test to run with iPhone 3GS,4,5 and iPad 2,3″ then Jenkins will run 5 runs of the tests in parallel?

To that end, as a post-JavaOne fun project, I developed a plugin that lists up all the iOS devices connected to Jenkins.

And just to show that we can list up various properties of this device:

I should be able to combine this with something like fruitstrap so that you can deploy apps from anywhere to any device connected to your Jenkins build farm.

What do you think? Shouldn’t this be interesting? Any thoughts?

jenkins, potd , ,

Jenkins Git Server Plugin

August 31st, 2012

Jenkins Git Server plugin is a so-called “library plugin”, which doesn’t offer any user-visible feature by itself, but instead enables other plugins to do something easily inside Jenkins. In case of Git server plugin, it allows other plugins to easily embed Git server functionality (via JGit) — create/manipulate Git repositories in the Jenkins server, expose it via SSH and HTTP transport for push/pull, and maintain local check out of those repositories.

As an example of how to use this plugin, I wrote Git userContent plugin. This plugin exposes the $JENKINS_HOME/userContent directory as a Git repository, and enables administrators to use git to push/pull changes and manage them with history.

In terms of code, there are two classes that plugins like git-userContent-plugin should be interested in.

One is HttpGitRepository, which represents Git repository access via HTTP. Typically you have some directory inside $JENKINS_HOME that houses the repository, then you subtype GitHttpRepository and override abstract methods to fill in the missing details. FileBackedHttpGitRepository is a convenient default implementation that simplifies this further. GitUserContentRepository in git-userContent-plugin is an example of using this class. This use also implements RootAction to bind this repository at http://server/jenkins/userContent.git, and I expect this combination to be fairly common.

The other class of interest is RepositoryResolver. Git server plugin adds necessary Jenkins SSH CLI hook for exposing Git repositories over SSH. The only missing link here is that when the client runs “git clone ssh://server/foo/bar/zot.git“, we need to figure out what repositories on the server corresponds to /foo/bar/zot.git, and that’s what the RepositoryResolver extension point does. The sample implementation in git-userContent-plugin will be hopefully self-explanatory. In this case, GitUserContentRepository is a singleton (because it’s RootAction), so we inject that and basically just delegate the calls to it.

I’m looking forward to seeing more plugins take advantages of this feature to expose data over Git repository. I think there’s a lot of interesting uses to it.

jenkins

Meetup “Automated testing & continuous deployment for mobile apps” tomorrow

August 29th, 2012

Mark Prichard from CloudBees, myself, and Matt Solnit from SOASTA will deliver a joint talk Thursday evening at Silicon Valley Cloud Computing Group.

Our part of the talk will take an iOS application and a backend Java server, then discuss how you can set up the pipeline of continuous build and test for them on Jenkins. The demo will be based on the CloudBees platform and we think it’s easier to do it that way than setting everything up on your own, but the setup should equally apply to your local Jenkins installation.

The part of the talk from Matt highlights SOASTA’s CloudTest product, which is basically a robot tha emulates user interaction on the device. My understanding of this is that it’s similar to Frank and KIF, but it comes with a recorder that lets you do “programming by example” — you manually interact with the test target app, and the recorder will turn that into a test scenario (Selenium IDE, anyone?)

Apple has locked down the iOS application development and distribution which makes the automated testing setup more painful than it should be, but because of that, I think a talk like this that shows the whole setup is worthwhile.

The event will be Thursday evening, at Yahoo! Campus in Sunnyvale. Please join us if you can. The event is free but requires RSVP. Beer and pizza are provided.

jenkins , ,

Jenkins User Conference Israel

July 1st, 2012

I’ll be visiting Israel this week, for Jenkins User Conference in Herzliya. I think this is my 3rd time visiting there, and I always enjoyed my visit and people over there (it really is an interesting place in many ways that everyone should visit once!) If you live in Israel, you can still register for the event!

You can see the planned talks, but there’ll be a lot of good “show and tell” talks where seasoned users share their experiences and setups to others, as well as more “emerging technologies” kind of talks, where people can get the idea of what’s cooking.

I’m planning to do a talk that contains a little bit of both; a bit about features that I developed for Jenkins that I use myself (some with my CloudBees hat on), then more about what the community has been working on in the core lately, and what I’d like to work on in near future.

It never ceases to amaze me that this piece of software called Jenkins has grown so much that we have 100s of people using it everywhere we go. Looking forward to meeting you all, and don’t forget to demand Jenkins stickers from me!

jenkins ,