Auto-discovering Hudson in the network

When you are writing an application that interacts with Hudson, it is often convenient to be able to auto-discover Hudson deployments on the same network. This improves the overall user experience.

There are several ways to do this — one way to do this, which is supported for the longest time, since 1.282, is via the UDP broadcast to port 33848. You send an empty UDP datagram to 255.255.255.255 port 33848, and all the Hudson instances on the same network will reply an XML fragment UDP packet to you, that looks like this:

<hudson>
  <version>1.354</version><!-- version of the Hudson -->
  <url>http://server/hudson/</url><!-- the top page of the Hudson -->
  <slave-port>12345</slave-port><!-- TCP port number for slaves and CLIs to connect to -->
  ... more elements may appear here ...
</hudson>

Starting 1.335, Hudson started using IP multicast socket to listen to this broadcast, so you can discover them by sending a broadcast packet to 239.77.124.213 instead of 255.255.255.255 — depending on how the network is configured, this can also reach beyond the same subnet.

Starting the upcoming 1.359, you can also discover Hudson via DNS multi-cast at “_hudson._tcp.local”. This also defines a convention that enables wide-area discovery of Hudson — such as “_hudson._tcp.infradna.com” for the Hudson deployed for infradna.com. The same version, url, and slave-port parameters are available as the key/value pairs associated with this DNS record, so you can find the actual URL of Hudson instances that way.

As we the engineers use more and more server applications for software development, more seamless integrations between them get more important, and I think the auto-discovery is a very important part of this.

Introducing InfraDNA, the Hudson company

As I wrote in my farewell note, I was working on starting a new company around Hudson. It took longer than I initially anticipated, but it’s finally open for business!

The company will provide two things; one is support, so that I can answer your questions and problem reports in a timely fashion, and the other is consulting, so that I can help you develop custom plugins, or provide on-site support to work on some tricky problems.

The name of the company is InfraDNA because I think of Hudson more as an infrastructure on which all kinds of server-side automation/tools can be built/deployed, and because I think this stuff is built into me (as in DNA) — when I look back my career as a software engineer, I always somehow seem to come back to tooling. (Plus, the domain name was available!)

Looking forward to hearing from you.

POTD: GitHub API for Java

My project of the day (or “POTD”) is GitHub API for Java — a library for accessing GitHub programmatically.

As the Hudson community is embracing plugins developed in Git more and more, I needed to interact with GitHub as a part of the community infrastructure automation. I did a quick Google search to locate existing implementations, but unfortunately I couldn’t find anything good. So I decided to just write my own. Thanks to a reasonable API design of GitHub and a good documentation, it was very easy to do so. The trick is to use the right library, which handles most of the JSON/Java databinding.

The library so far only covers the part of the GitHub API that I care about, which is a small subset of the entire GitHub API. But hopefully this library is easy enough to extend so that other people can add the remaining APIs. The source code is available in GitHub.

Hudson console markups

Despite all the report comprehension in Hudson, such as JUnit, PMD, FindBugs, etc., log files still hold a special place in terms of capturing what has really happened. Hudson does a bit of AJAX in this space to let you follow output as it comes, but the log is basically just a plain text that doesn’t really have structures.

But that is changing. One of the recent improvements in Hudson is the infrastructure and extension points for Hudson (and its plugins) to mark up the console output to improve interactivity and do some cool stuff.

I prepared two kinds of extension points for this. One is the ability to scan the console output line by line and add arbitrary markup to it. This can be used for context-independent markup, for example to turn URLs into hyperlinks, look for keywords like “ERROR”, that sort of things.

The other kind is more interesting, where we can place anchors (I call them ‘notes’) at arbitrary points during the output, and those notes can then in turn generate markups. This enables highly context sensitive markups, which I think has a lot of potential.

For example, I started putting a note for every Ant target that gets executed during the Ant execution. I can use this to generate outline for the console output, so that you can jump to the interesting targets, or move up/down to next target very quickly. For simple build scripts, I can let users click the target name and jump to its definition in the build script.

Another place I do this today is when Hudson reports an exception. I can make a stack trace foldable so as not to overwhelm users, and I can also hyperlink each stack trace element to its source file, as a way to encourage people to start hacking Hudson. Or if a build fails, I can present an UI that gives you actions that you might want to take — 1. edit config, 2. rebuild, 3. report to the admin, etc.

With Maven, where Hudson puts a little spying agent inside the Maven process, I can do even better. For example, wouldn’t it be nice if you can hide all the “[INFO]” message with one mouse click? How about a navigation from compilation failure reports to source files? Or if you have an outline of modules that were built and jump to them quickly?

If you are an user, this is just a sneak preview into what will come. If you are a plugin developer, think about all the things you might want to do with this mechanism!

POTD: Custom Access Modifier

My project of the day (or "POTD") is Custom Access Modifier — an annotation and an enforcer that lets you define application-defined custom access modifiers,

So let me explain this a bit more. Say you have a library that people use, and say you are thinking about deprecating one of the methods. Yes, you can just put @Deprecated, but that doesn’t actually prevent people from continuing to use them. This is where you can put the custom access modifier, like this:

 
public class Library {
    @Deprecated @Restricted(DoNotUse.class)
    public void foo() {
        ...
    }
}

This causes compilation to fail for new source files that try to call the foo method. But at the resulting class file still contains the method, so existing applications continue to work. As per the JVM spec, this contraint enforcement is strictly in the user land and thus voluntary, and at the runtime there’s no check nor overhead.

Or say you have a "public" class that’s never intended to be used outside your library? Not a problem.

 
@Restricted(NoExternalUse.class)
public class FooBarImpl {
    ...
}

In the first version, I packaged the enforcer as a Maven mojo, but it should be trivial to write an Ant task or CLI. A real usability improvement is if this can be done as JSR-269 compatible annotation processor, but unfortunately the enforcer needs bytecode level access to the source files being compiled, and I don’t think JSR-269 gives me that, which is a pity.

The real flexibility here is that you can define your own access restrictions, not just using those that I provided out of the box.

The reason I came up with this is to better assist the feature deprecation in Hudson. With 6+ years of the code history, there are a fair amount of deprecated code in the foundation. We’d eventually like to remove them, but we can’t just delete them all the sudden — there might be plugins using them out there. But with this plugin, I can actually make sure that plugins are not using those deprecated features that are candidates for removal.

I hope you’ll find this tool useful. The source is on GitHub.

Hello

For the longest time I haven’t really done anything about kohsuke.org, but as I left Sun/Oracle, I decided to put a bit more effort into it. So this is the new home.

For the time being, I plan to post my blogs both on java.net and here.