Archive

Archive for the ‘potd’ Category

POTD: ExceptionInInitializerError logger

January 28th, 2015

It’s been a while I’ve done a project of the day, but here it is, the fruit of my yak-shaving today.

The problem I was trying to solve today was java.lang.NoClassDefFoundError: Could not initialize class Xyz. When a Java class fails to initialize, the first attempt to do that causes ExceptionInInitializerError, but subsequent attempts to use that class results in this rather unhelpful java.lang.NoClassDefFoundError: Could not initialize class Xyz without the chained exception.

This problem has been rpeorted to Java for years, but probably JavaSE people doesn’t understand how painful this is in a large modular system, where the initial exception can be reported in so many places — such as one of the 1000 builds you’ve done today, or in an HTTP response to somebody, stderr, logging, or getting swallowed by empty catch block.

So I wrote a little Java agent that uses java.util.logging to log every ExceptionInInitializerError at the point of instantiation. In this way, even on a server, you have one place you can go to check for all errors of this kind. Through j.u.l, you can write a custom Handler to report errors elsewhere, if you want to.

The number of people who will find this tool useful would be probably small, but I hope they’ll really appreciate this little gem. May Google let them find this page.

potd ,

POTD: random but meaningful name generator

April 6th, 2014

I’m working on the automated blackbox acceptance tests for Jenkins, where I often need to generate random unique names. The code has been using random number generator to generate such names, but as I was debugging test failures, it became painful to remember those random names.

For example, a test might create two new jenkins jobs “random_name_155230″ and “random_name_137204″. Now which one was supposed to be the upstream and which one is downstream? Aside from a few exceptions, humans are generally not good at remembering those random numbers.

So I thought it’d be a lot better if these names are more meaningful, like “constructive_carrot” or “flexible_designer”. That is, if I have a decent sized corpus of N English adjectives and M nouns, I can generate NxM unique names (and induce a few chuckles to whoever see the generated names.)

After a bit of googling, I came across wordnet, and I took a subset of its corpus to come up with a small library that generates human-friendly random names. It has about 600 adjectives and 2400 nouns, resulting in 1.5 million unique names before the generator wraps around.

You’d use this library like this:

RandomNameGenerator rnd = new RandomNameGenerator(0);

while (true)
    System.out.println(rnd.next());

The code is under the BSD license with no advertisement clause, and the library is on Maven Central. Hope you find it useful.

potd , , ,

POTD: Application configuration via Guice binding + Groovy

March 1st, 2014

Often I write my applications with Guice. I also often want to make those applications configurable externally. For example I might inject username and password for that app to talk to another app, I might configure some timeout value, and so on. I make these configuration values available in Guice, so that I can access them wherever I need them. All of this is pretty common in many other places, I’d imagine.

Given that all I’m doing here is to pass configuration values from left to right, I thought it’d be nice if I can write configuration directly as a Guice module by using Guice binder EDSL. Then I won’t have to parse and translate these configuration any more.

And that became my project of the day.

This little library allows you to write Guice binding definitions in a text file:

timeout = 3
bind Payment named "customer" to VisaPayment

From your program, you use GroovyWiringModule to load this configuration file:

Module config = new GroovyWiringModule(new File("/etc/myapp.conf"));
Injector i = Guice.createInjector(
  Modules.override( ... my application's modules ...)
    .with(config))

The end result is that the above script gets translated into the following binding:

bind(int.class).annotatedWith(Names.named("timeout")).toInstance(3)
bind(Payment.class).annotatedWith(Names.named("customer")).to(VisaPayment.class)

Using Groovy as the host language for DSL has other benefits. If you are using system properties or environment variables to configure something, you are basically stuck with strings as the only representation of the configuration. With Groovy, I can create a relatively complex object and bind them, or even put some logic to further obtain values from elsewhere:

bind Payment toInstance new VisaPayment(
  cardNumber: "1234-5678-9012-3456",
  expiration: new Date(System.currentTimeInMillis()+TimeUnit.DAYS.toMillis(30),
  cvv: new URL("http://secret.server/cvv").text) 

With the functionality in Guice to override definitions in one module by another, I can also even override bindings defined in programs, for example to get more logging, add a filter, etc.

potd , , , ,

POTD: cucumber annotation indexer

February 27th, 2014

Cucumber for Java requires that you specify the packages in which your step definitions exist. At runtime, cucumber uses some hack to try to list all the classes in this package (it’s a hack because class loaders never really support the listing operation), loads them one by one, and finds those that have step definition annotations like @When and @Then. This is both poor user experience (can’t you just find my step definitions!?) and poor performance (loading all the classes under a package is expensive.)

So I wrote a library that offers a much better alternative. It uses annotation indexer to create an index of step definitions and hooks at compile time. Thanks to JSR-269, this happens automatically on Java 6 and later. With the index in /META-INF/annotations, runtime can load all the step definitions quite efficiently.

The library contains a Backend implementation, so you should be able to just add it to your project dependency, and cucumber should automatically find this (and thus all your step definitions and hooks.)

By the way, this horrible technique of scanning jar files, listing class files, and finding annotations from there is unfortunately commonly seen in many other libraries. This was a necessary evil in the days of Java 5, but it should really die in this day and age. If you realy on the classpath scanning, please switch to annotation indexer, which provides the backbone functionality of this POTD.

potd , , , ,

POTD: no more tears

December 14th, 2013

In a modular Java program or in a large Java project that has lots of dependencies, you often end up a version of library that’s different from the version used to compile the code.

This often results in LinkageError, where a method/field that was present when the code was compiled do not exist any more in the version being loaded at the runtime. This restriction applies to seemingly trivial safe changes, such as changing the return type of a method to the subtype of what it used to be.

Previously, the only way to deal with this is not to remove any signatures that matter. In other words, you count on library/module developers to be more disciplined. Over the time, Java programmers have accepted this as a way of life, but there are some notorious offenders (Guava and ASM, I’m looking at you.) Besides, it makes it difficult to evolve code.

I have dealt with this in multiple different ways in the past.

The bridge method injector is an example of static compile-time transformation. This kind of tool is non-intrusive to the users of a module, which is good, but it’s still a tool for library developers to be diligent, and processing at compile time means it has only limited information to operate on.

The bytecode compatibility transformer is an example of runtime transformation. This has a lot more information to let it do the right transformation, but modifying class files on the fly requires a custom classloader, which limits its applicability.

On the way back from my recent trip, I realized there is the 3rd way to achieve the same effect — invokedynamic. You see, invokedynamic is really just a mechanism of deferring the linking to the runtime. This allows me to combine the benefit of two approaches. I can transform class files at the compile time without really deciding how the references are linked. Then at runtime, I can decide how they actually get linked but without a need of runtime transformation. The only downside is that it requires Java7.

But in any case, I thought the idea was clever, so I implemented it as my “project of the day”. Please let me know what you think.

potd , , ,

POTD: StopForumSpam API for Java

December 22nd, 2012

There’s an on-going spam problem in Jenkins wiki. We have capture, but either that was broken, or more likely, spamming is done manually.

One of the suggestions was to integrate StopForumSpam service. To that end, I wrote a simple Java client library for their API. The usage would be something like this:

    for (Answer a : new StopForumSpam().build().ip(ip).email(email).query()) {
        if (a.isAppears()) {
            LOGGER.warning("Rejecting, likely spam: "+a);
            throw new UserError("Due to the spam problem, we need additional verification for your sign-up request. Please contact jenkinsci-dev@googlegroups.com");
        }
    }

My hats off to them for providing a great service, and here is hoping that this will help make the dent. We’ll see.

potd

POTD: submit a patch to Jenkins, and let him test it for you

October 8th, 2012

Here’s my 2nd after-JavaOne “project of the day” Jenkins plugin. This has been in the back of my mind for quite some time, but it took this gentleman to grill me on this feature during JavaOne for me to finally put it together — so thank YOU for doing that. although I didn’t catch your name.

The plugin is called the “patch parameter” plugin. This plugin lets you submit a patch when scheduling a build, and that gets applied to the checked out source tree before a build would commence. Any failure to apply a patch will result in a build failure.

This plugin can be used for a “pre-tested commit” workflow. You can work on a change locally, have the diff tested on the server, then if you are satisfied, you can commit it. I can imagine this would be also an interesting building block for integration with code review tools.

I should also note that we have the Subversion Merge plugin for a different approach to a similar problem, and distributed VCS can generally do this better (for example I do one for Git in CloudBees Validated Merge plugin.)

jenkins, potd , ,

POTD: iOS device connector plugin (cont’d)

October 7th, 2012

Today, I wrapped up the project I started Friday and released iOS device connector plugin.

In addition to listing all iOS devices, this plugin lets you deploy IPA files from anywhere (in Jenkins build via a build step or outside via CLI) to any of the connected iOS devices.

There are still some loose-ends that I’d like to get some feedback on:

  • You should be able to select a device without electing a specific UDID, and I’m curious what kind of criteria people needs here. Do you need to be able to say “deploy my app on iPad2 with 16GB memory”, or is “deploy my app to an available iPad” suffice?
  • For this to be useful for automated test execution. I think we need to be able to launch the app (and I assume the tests embedded in the app itself will run and shuts itself down in the end?) How do people do it?
  • Don’t we need to inject the coordinates of the actual device the app was deployed to, so that your build can do more stuff with it?
  • What are other interesting operations to devices. File access? Screenshot?

Anyway, the plugin is already released. I hope you find it useful.

jenkins, potd

POTD: iOS device detection in Jenkins

October 5th, 2012

I was talking to my colleague Mark Prichard about mobile development with Jenkins, and I came up with this idea.

If you are doing iOS app testing with real devices, you need to tether the device with a computer so that you can push the app-to-be-tested to the device. In a local development environment, you’d naturally do this by hooking up a device to your laptop.

But in the CI environment, we can do better. Imagine if you have a Jenkins master connected to a dozen Mac minis, where each Mac mini is then connected to various iOS devices. There’s nothing stopping Jenkins from listing up all the devices connected to all the slaves. Then based on this information, Jenkins can lease devices to builds.

Say someone wants to run a test somewhere. Jenkins can find an unused device and push it, then inject the coordinate of that device to the build. Or how about the axis in a matrix project that allows you to say “I need this test to run with iPhone 3GS,4,5 and iPad 2,3″ then Jenkins will run 5 runs of the tests in parallel?

To that end, as a post-JavaOne fun project, I developed a plugin that lists up all the iOS devices connected to Jenkins.

And just to show that we can list up various properties of this device:

I should be able to combine this with something like fruitstrap so that you can deploy apps from anywhere to any device connected to your Jenkins build farm.

What do you think? Shouldn’t this be interesting? Any thoughts?

jenkins, potd , ,

POTD: Confluence static cache generator plugin

June 22nd, 2012

I’m not sure about your Confluence, but my Confluence was dog slow. Page rendering regularly took a second or two, or even worse. That’s why I wrote this plugin.

This Confluence plugin generates static HTML files out of your Wiki pages. It happens every time when someone updates a page, post a comment, add a label, and so on. Anything that can affect the way the page looks in Confluence, and the cache gets regenerated.

Unlike Auto-export plugin, this plugin doesn’t try to apply a different theme, or help you make them look like non-Wiki sites, because the goal is the opposite. I want to make Confluence look faster, as opposed to generating website out of Confluence. To this end, HTML files generated by this plugin is a straight capture of what Confluence is rendering.

These generated files can be then served from frontend Apache reverse proxy. With the use of `mod_rewrite`, Apache will act as a smart caching proxy; it’ll serve static HTML files if they exist, and otherwise it’ll forward the requests to the backend Confluence.

The end result is that your visitors won’t even notice that your Confluence is running with cache. They see the same-old Confluence UI, and the same URLs will keep working. The only difference is that pages magically load a lot faster.

See the live instance to get the feel of how this cache works mostly transparently, and if you want to try this on your own Confluence, the code is here.

potd , ,