Archive for January, 2013

Jenkins commit activities in 2012

January 25th, 2013

Stephen is having fun with Git and R, and I saw another person creating a similar comparison (although I’m not sure where the credit should go on that one.)

Both charts are great, but I also noticed that their Y-axis aren’t consistent. So if people aren’t careful they might not see Jenkins as favorably as it actually is. So here is the graph, comparing # of commits in the year 2012. They both only includes the core and in the master branch:

On average, we have 41.2 commits per week, compared to 9.7 on the other side. If you want the raw data, here is the Excel file.

hudson, jenkins

JavaFX needs to be a new edition of Java

January 20th, 2013

Lately, there has been a number of security vulnerabilities reported in Java. The latest one is reported just after a few days of JavaSE 7u11, which by itself a response to another vulnerability. It’s so bad to the point that people are being asked to uninstall Java (yes, just in the browser, but let’s face it, it’s way easier to just uninstall it entirely than to disable the Java plugin from all the browsers.)

I think this issue needs to be addressed at a deeper level pretty quickly, or else what’s still left of the client-side Java would be dead soon (if you think it’s already dead, then there’s nothing else to see in this post, so please move on.)

In fact, one could even argue that the entire Java platform is at risk — people (especially our users and those who are studying programming) do not understand that these issues are the sandbox breaches and therefore do not affect the server-side Java, nor embedded Java.

I think the main lesson should be that the sandbox model of Java is unmaintainable. It’s not just a few isolated bugs here and there, but it’s a structural problem.

For those of you who aren’t very familiar with Java, basically Java sandbox model is that:

  • At runtime the entirety of Java core libraries are present (the exact same code you run on the server-side, including such abilities as forking processes, making network connections to anywhere, and accessing files)
  • Code gets associated with the “code source” information to indicate if it’s trusted or not
  • Library code (like one that accesses files) checks if the caller is trusted or not, and if not certain operations aren’t allowed.

The check in step #3 is implemented in a Java code (called SecurityManager), and that’s where the problem is. All the recent vulnerabilities in Java basically involves runtime reflection to replace the effective SecurityManager instance. This forces the reflection library to be inside the protection wall built by the security layer, which in turns forces many libraries in JavaSE to be inside the wall (such as JMX, which is used for the recent exploit, as well as XML stuff, which I was personally involved in.)

At this point, the protection wall is so long and windy that it’s impossible to defend. And I think it’s really all because we decided to protect reflection.

IMHO, a much simpler way to let untrusted code run is just to enforce checks on the native side, and simply stop trusting anything that runs as Java byte code. After all, we really need to just check stuff that interacts with the outside world, and every one of those has to be done by calling into native code. There’s very few of these calls. In this way, neither JMX nor XML APIs can be exploited, because they simply become just another untrusted code (that just so happen to be available all the time.) I’m no Flash expert, but IIUC, this is how Flash security model works.

In other words, create a new edition of Java, by adding one more to the ME/SE/EE mix. It is a new runtime environment that just shares the same virtual machine and the subset of core libraries. I think this is already consistent with the way “JavaFX” is marketed. While technically speaking it’s just a fancy UI library, it’s really marketed as an RIA platform along ME/SE/EE.

With the sophisticated existing IDEs, static typing, people’s familiarity, and the formidable existing ecosystems, I think Java (VM and the byte code in particular, and to somewhat less extent the language) is still incredibly useful to the large-scale client side development. Especially so if a bit more can be added, such as a better interoperability with DOM and other HTML5 APIs. While JavaScript is getting better day by day, there are still many things Java does better.

Imagine being able to define a large application into modules, with interfaces to define boundaries, mature module systems, efficient delivery and caching mechanisms. Debugger support, multi-threading, runs outside the browser for unit testing, programming in Ruby or Groovy … you get the idea. Think of it as GWT except translation, if you may.

As an avid Java fan, here’s my plea to Oracle — I want this for my Christmas present this year.


On the road for the rest of the month

January 10th, 2013

I’m really excited to kick off my 2013 with a tour around the world for Jenkins.

The first stop will be in Tel Aviv, where I’ll be doing training with AlphaCSP. This has sold out, but AlphaCSP will be delivering this training again in the future. I’ll then head to London for the Jenkins User Event in London, with James Nord (of the m2 release plugin fame) and Andrew Phillips from XebiaLabs. This one has also sold out, but I’ll be recording the talks and posting them online. After tha, I’m in Munich to deliver another Jenkins training with ObjectBay. Again the same deal here — it’s sold out but they’ll be scheduling repeats later.

I’ll then be back to my home for a few days, then off to Seoul for the first-ever Jenkins meet-up in Korea. I’m really excited about this, and I’m trying to kick-start a local community, which is invaluable in this region of the world. If you are in the area, please come join us, and get to know other people in the Jenkins community.

Then I head to Tokyo to an user meet-up (no registration page is up yet, please check back our website later), and a couple more trainings (this one has more seats available.)

From there I’m flying to Brussels for FOSDEM. They gave me a keynote slot (yay!), and Tyler is organizing the testing and automation devroom. A number of Jenkins people will be there, and we’ll have a table for two days. This will be exciting!

And in my last leg of the journey, I’ll be at JFokus in Stockholm. They kindly gave me a long 3 and half hour slot to talk about Jenkins in depth, aside from more typical 50 mins talking slot. I’m looking forward to this new format.

And when I finally get back to home, I suspect I need some down time to recover.

If you are in the area nearby, and grab some stickers or want to say hello, please let me know! And my apologies in advance for the delay in responding e-mails and other disruptions.

jenkins , , , ,

The other side of forking and pull requests

January 4th, 2013

Charles Nutter of JRuby fame had this tweet yesterday:

And this touches on something I’ve been thinking for a long time.

My experience mostly comes from the Jenkins project, which is one of the open commit policy projects. Everyone gets to be a committer just by asking, and that grants them access to all the plugins. This policy is deep in our culture that we even have an IRC bot that the project people use to add new committers.

This is not necessarily a good idea for every project, but it certainly served us well, especially back in the days when we had the entire source code in the Subversion repository. During that time, for people to make changes to the plugins, they really needed to be a committer — maintaining a custom patch set on top of an external Subversion repository is a royal pain in the back side. The open commit policy means anyone interested in committing can join the project without a fuss, and once they become a contributor and start commiting code, they feel like being a part of the project, and that feeds fresh blood into the Jenkins developer community.

While I totally believe in the technological superiority of Git as a version control system over Subversion, in my view, the fork and pull-request driven world of Git hasn’t been a complete win for the Jenkins project.

Forking and pull requests do encourage contributions by eliminating certain barriers to start hacking on someone else’s project. Whereas it was quite painful to maintin patches externally to someone else’s source code in Subversion, Git makes this trivial with branching and occasional rebases or merges. This part is a good thing.

On the other hand, this ease of maintaining your own versions reduces the incentive to bring your changes back to the community, to some extent. After all, you’ve already scratched your itch and it works on your machine. Why bother trying to push the code back and go through the trouble of writing tests and documenting the change, convincing others about the design, and arguing why that change was needed to begin with. You can see this in a number of forks people created that never turns into as pull requests, and I’ve done this a number of times myself.

An even bigger problem in my mind is that by allowing people to contribute by just sending pull requests, it encourages the “throw the stuff over the wall” way of contributing. Sometimes people can do stuff without telling us anything upfront, and you as a receiver would be hard-pressed to turn it down, given that someone spent so much effort in it. Sometimes when we request additional changes, they never come back. It makes the collaboration and communication after the fact, not before the coding. And it also makes the pathway to the committer bit unclear. How do we convert people from the side sending pull requests to the side receiving pull requests? If we have to ask them one by one it won’t scale.

The situation gets worse on plugins, where there may not be an active maintainer at the moment. We’d like those who are sending us pull requests to take over the maintainership, and use their best judgement to evaluate the changes. But when GitHub makes them feel like the proper way to contribute is by sending a pull request, and if your pull requests do not get any attention, I’m not sure how they are supposed to realize that the plugin needs a new owner. Back in the days of Subversion, the repository alone was enough to be a center of gravity of the project. It attracts new developers (so long as we let them be, aka open commit policy) and it’s almost self sustaining. Now it takes more than just the repository.

In any case, I just wanted to point out that the impliations of forking and pull requests are bit more nuanced.

P.S. I’ve been long meaning to write a bot that patrols for such unattended pull requests, and encourage them to become committers, but haven’t had a chance to do so yet. I still intend to do this, and we’ll see how it goes.

jenkins , , ,