Archive

Archive for the ‘potd’ Category

POTD: Groovy Sandbox

April 29th, 2012

I posted the other day about Groovy’s SecureASTCustomizer and how it is harmful. In the end of that post, I suggested that doing the check dynamically would work better.

So in this installaition of “Project Of The Day”, I went ahead and implemented it. The result is Groovy sandbox.

My main idea is to confine the sandboxed script into its local object graph. The script should be allowed to mutate this graph all it wants, but it’ll be only allowed to include objects of known whitelisted “safe” types (such as String, List, Date etc.), and a few known safe instances, which acts as a bridge between the sandbox and the rest of the world. These bridge objects would have to be written carefully.

To use this, you have to add SandboxTransformer to your CompilerConfiguration first:

def cc = new CompilerConfiguration()
cc.addCompilationCustomizers(new SecureTransformer())
def binding = new Binding();
binding.robot = robot = new Robot();
def sh = new GroovyShell(binding,cc)

And now any script compiled via the resulting shell object will be sandboxed.

When a sandboxed script executes, all of the following operations are intercepted.

  • static/instance method invocation foo.bar(....)
  • object allocation new Foo(...)
  • property access and assignment zot=foo.bar / foo.bar=zot
  • attributes access and assignment zot=foo.@bar / foo.@bar=zot
  • array access and assignment zot=foo[bar] / foo[bar]=zot

To examine those calls and reject some of them, create your own implementation of GroovyInterceptor and registers it to the thread before you start executing the script:

def sandbox = new RobotSandbox()
sandbox.register()
try {
    sh.evaluate("robot.leftArm.move()")  // this is allowed to complete
    sh.evaluate("robot.selfDestruct()")  // no!
} finally {
    sandbox.unregister()
}

See the robot example for a complete example.

Now let’s see if I can get some feedback from real Groovy experts, and see if they’d be willing to take this into Groovy itself…

potd , ,

POTD: Package renamed ASM

March 3rd, 2012

Today’s project of the day is a package renamed ASM library.

I previously wrote about a problem in otherwise quite useful ObjectWeb ASM library. Namely, it breaks backward compatibility in such a way that badly breaks apps/libraries that use them. In that post, I wrote about two proposals to fix the pain point. One is to include debug information, which has been fixed starting 3.x. But the other package renaming hasn’t been addressed in the last 2 years.

This has been in the back of my head, but it never came high enough until recently I had another NoSuchMethodError caused by ASM3. One of the servlet containers shipped ASM3 and it broke Jenkins that bundles ASM2. Between this and ASM4 release for JavaSE 7, which will likely gain popularity over time, I finally decided to fix this problem once and for all, in a way that everyone else can reuse.

The solution, as explained in the original post, is to put each major ASM version in its unique package name. I pakage-renamed ASM2 in org.kohsuke.asm2, ASM3 in org.kohsuke.asm3, and ASM4 in org.kohsuke.asm4. The package name only contains the major version because I trust the ASM developers to maintain compatibility between minor releases (and I believe they’ve maintained this thus far.)

These artifacts are available in org.kohsuke:asm2:2.2.3, org.kohsuke:asm3:3.3.0, and org.kohsuke:asm4:4.0 — these are packaged renamed by jarjar and I tested them somewhat to make sure it’s not downright broken.

If library A depends on asm2 and library B depends on asm3, and someone else uses both A and B, everything will work fine because asm2 and asm3 are in the different pcakages. If A depends on one version of asm3 and B depends on another version of asm3, then the transitive dependency resolution will pick up the newer version and both will work (or you end up implicitly picking up one version over another, and you don’t enjoy the latest bug fixes, but at least it won’t die with LinkageError.)

When you search “asm3″ in Maven central, you see a large number of renamed ASM3 in various projects. Hopefully that madness will stop now!

The other interesting thing about this effort is that I’ve used Gradle to package rename them. Lately I’ve been using Gradle for publishing transformed artifacts like these to a Maven repository, and I like it a lot. But more about that in another post.

potd, Uncategorized

POTD: checking package name for Windows reserved words

March 1st, 2012

Today’s project of the day is a little tool that checks problems in your package names that can bite you on Windows.

For backward compatibility with MS-DOS (which had ended its life more than 15 years ago!), Windows doesn’t allow you to create files/directories of certain names, such as “AUX” or “PRN” (see Microsoft Knowledge Base.)

This can bite those of us who develop on Unix — when you accidentally create a file or a directory of this name (say in the package name), it’ll be all right for you, but your colleagues who use Windows will not be able to build them. Worse yet, if none of your colleagues use Windows either, this problem can get unnoticed until you ship/test the code.

So I wrote a little tool that proactively checks for this problem. It’s an annotation processor that kicks in automatically, so you just need to add it in your classpath when you compile your source files. If you use a build system that can handle transitive dependencies (such as Gradle or Maven), you just need to use it in one of your libraries, and all the code that uses your library also benefits from this check.

My hats off to Microsoft for their rigorous commitment to the backward compatibility, and just one more reason why you should be always testing your Java program on multiple platforms…

potd , ,

Jenkins ChromeDriver plugin

December 28th, 2011

I released Jenkins ChromeDriver plugin that auto-installs chromedriver on all slaves. Here is why this is useful.

ChromeDriver is needed to run WebDriver-based tests with Chrome. Unfortunately this is a native program, so in a heterogenous Jenkins, you need to make sure you have the right binary available on each of the slaves. While that’s not hard, it’s awfully tedious. This plugin fixes that by automating it.

This plugin would be a nice companion to the recently upgraded Selenium Grid plugin that automatically runs Selenium Grid on top of your Jenkins slaves. When you combined, you just install two plugins to the Jenkins master, and without touching slaves at all, you got the whole cluster instantly ready for Selenium2 WebDriver testing. How is that?

jenkins, potd ,

POTD: bridge method injector

August 7th, 2010

I was working on Hudson yesterday which led me to develop this little tool called Bridge method injector.

When you are writing a library, there are various restrictions about the kind of changes you can make, in order to maintain binary compatibility.

One such restriction is an inability to restrict the return type. Say in v1 of your library you had the following code:

public Foo getFoo() {
    return new Foo();
}

In v2, say if you introduce a subtype of called FooSubType, and you want to change the getFoo method to return FooSubType.

public FooSubType getFoo() {
    return new FooSubType();
}

But if you do this, you break the binary compatibility. The clients need to be recompiled to be able to work with the new signature. This is where this bridge method injector can help. By adding an annotation like the following:

@WithBridgeMethods(Foo.class)
public FooSubType getFoo() {
    return new FooSubType();
}

… and running the bytecode post processor, your class file will get the additional “bridge methods.” In pseudo-code, it’ll look like this:

// your original definition
@WithBridgeMethods(Foo.class)
public FooSubType getFoo() {
    return new FooSubType();
}

// added bridge method
public Foo getFoo() {
    invokevirtual this.getFoo()LFooSubType;
    areturn
}

Such code isn’t allowed in Java source files, but class files allow that. With this addition, existing clients will continue to function.

In this way, you can evolve your classes more easily without breaking backward compatibility.

For more about how to use it in your Maven project, the project website.

potd

POTD: GitHub API for Java

April 18th, 2010

My project of the day (or “POTD”) is GitHub API for Java — a library for accessing GitHub programmatically.

As the Hudson community is embracing plugins developed in Git more and more, I needed to interact with GitHub as a part of the community infrastructure automation. I did a quick Google search to locate existing implementations, but unfortunately I couldn’t find anything good. So I decided to just write my own. Thanks to a reasonable API design of GitHub and a good documentation, it was very easy to do so. The trick is to use the right library, which handles most of the JSON/Java databinding.

The library so far only covers the part of the GitHub API that I care about, which is a small subset of the entire GitHub API. But hopefully this library is easy enough to extend so that other people can add the remaining APIs. The source code is available in GitHub.

potd

POTD: Custom Access Modifier

April 12th, 2010

My project of the day (or "POTD") is Custom Access Modifier — an annotation and an enforcer that lets you define application-defined custom access modifiers,

So let me explain this a bit more. Say you have a library that people use, and say you are thinking about deprecating one of the methods. Yes, you can just put @Deprecated, but that doesn’t actually prevent people from continuing to use them. This is where you can put the custom access modifier, like this:

 
public class Library {
    @Deprecated @Restricted(DoNotUse.class)
    public void foo() {
        ...
    }
}

This causes compilation to fail for new source files that try to call the foo method. But at the resulting class file still contains the method, so existing applications continue to work. As per the JVM spec, this contraint enforcement is strictly in the user land and thus voluntary, and at the runtime there’s no check nor overhead.

Or say you have a "public" class that’s never intended to be used outside your library? Not a problem.

 
@Restricted(NoExternalUse.class)
public class FooBarImpl {
    ...
}

In the first version, I packaged the enforcer as a Maven mojo, but it should be trivial to write an Ant task or CLI. A real usability improvement is if this can be done as JSR-269 compatible annotation processor, but unfortunately the enforcer needs bytecode level access to the source files being compiled, and I don’t think JSR-269 gives me that, which is a pity.

The real flexibility here is that you can define your own access restrictions, not just using those that I provided out of the box.

The reason I came up with this is to better assist the feature deprecation in Hudson. With 6+ years of the code history, there are a fair amount of deprecated code in the foundation. We’d eventually like to remove them, but we can’t just delete them all the sudden — there might be plugins using them out there. But with this plugin, I can actually make sure that plugins are not using those deprecated features that are candidates for removal.

I hope you’ll find this tool useful. The source is on GitHub.

potd , ,