Archive

Archive for the ‘Uncategorized’ Category

Support API freedom

April 5th, 2013

I was reading this article from Steve and Sacha about the API copyrightability, and found myself in a violent agreement. If you haven’t read it, I highly recommend it.

For those of you who haven’t been following the tech news, the issue at hand is Android — Google neatly side-stepped Java’s compatibility requirements by introducing a new runtime/VM and said that Android is not Java. Oracle sued Google claiming that the Java API is a copyrightable material, and that Google can’t just create a whole new implementation that’s API-compatible with Java. Oracle lost the case, but now Oracle is appealing, and they are garthering legacy vendor friends to argue that API not copyrightable is bad for economy.

But wait, surely more competition is bad for them vendors, but what about the instant gain we the developers got when Android came along, in becoming instantly productive in this entirely new platform?

Looking at the comment secion of the article, I was bit disappointed that some people saw this only as a storm in a teacup, or that this is an issue only about Java, and their favorite programming ecosystem (C#, Ruby, …) are OK. But it’s quite a contrary. If the appeal is successful, it has a broad implication on all sorts of APIs. As they say, first they come for the communists, and you think you are safe, but by the time they come to you, it might be too late!

Take Mono for example. Sure, C# and CLI are under Microsoft Community Promise. But what about the vast APIs in the .NET Framework, which is necessary for writing any meaningul application? What about all the Win32 APIs that Wine implements? Or how about Eucalyptus implementing the Amazon Web Services API? Sure, they might be in a good relationship now, but what if IBM acquires Eucalyptus and started a cloud offering of the same API?

As a developer I benefit every day from the compatibility and being able to migrate from one vendor to another without losing everything. And when I look back at PC/AT, x86 instruction sets, Java EE APIs, and so on, I truly believe that the openness is good not just for us the developers but for the broader economy as a whole.

So after reading the article, I felt like I wanted to help the cause and voice my support, but I wasn’t sure how — I’m just a developer and not a lawyer. So I created a White House petition. Not so much because I expect the White House to do something about it, but it’s a good enough neutral petition site that hopefully people feel safe enough to join. If you agree with the cause, please join the petition and help spread the words, so that our voices get heard.

Uncategorized , ,

JavaFX needs to be a new edition of Java

January 20th, 2013

Lately, there has been a number of security vulnerabilities reported in Java. The latest one is reported just after a few days of JavaSE 7u11, which by itself a response to another vulnerability. It’s so bad to the point that people are being asked to uninstall Java (yes, just in the browser, but let’s face it, it’s way easier to just uninstall it entirely than to disable the Java plugin from all the browsers.)

I think this issue needs to be addressed at a deeper level pretty quickly, or else what’s still left of the client-side Java would be dead soon (if you think it’s already dead, then there’s nothing else to see in this post, so please move on.)

In fact, one could even argue that the entire Java platform is at risk — people (especially our users and those who are studying programming) do not understand that these issues are the sandbox breaches and therefore do not affect the server-side Java, nor embedded Java.

I think the main lesson should be that the sandbox model of Java is unmaintainable. It’s not just a few isolated bugs here and there, but it’s a structural problem.

For those of you who aren’t very familiar with Java, basically Java sandbox model is that:

  • At runtime the entirety of Java core libraries are present (the exact same code you run on the server-side, including such abilities as forking processes, making network connections to anywhere, and accessing files)
  • Code gets associated with the “code source” information to indicate if it’s trusted or not
  • Library code (like one that accesses files) checks if the caller is trusted or not, and if not certain operations aren’t allowed.

The check in step #3 is implemented in a Java code (called SecurityManager), and that’s where the problem is. All the recent vulnerabilities in Java basically involves runtime reflection to replace the effective SecurityManager instance. This forces the reflection library to be inside the protection wall built by the security layer, which in turns forces many libraries in JavaSE to be inside the wall (such as JMX, which is used for the recent exploit, as well as XML stuff, which I was personally involved in.)

At this point, the protection wall is so long and windy that it’s impossible to defend. And I think it’s really all because we decided to protect reflection.

IMHO, a much simpler way to let untrusted code run is just to enforce checks on the native side, and simply stop trusting anything that runs as Java byte code. After all, we really need to just check stuff that interacts with the outside world, and every one of those has to be done by calling into native code. There’s very few of these calls. In this way, neither JMX nor XML APIs can be exploited, because they simply become just another untrusted code (that just so happen to be available all the time.) I’m no Flash expert, but IIUC, this is how Flash security model works.

In other words, create a new edition of Java, by adding one more to the ME/SE/EE mix. It is a new runtime environment that just shares the same virtual machine and the subset of core libraries. I think this is already consistent with the way “JavaFX” is marketed. While technically speaking it’s just a fancy UI library, it’s really marketed as an RIA platform along ME/SE/EE.

With the sophisticated existing IDEs, static typing, people’s familiarity, and the formidable existing ecosystems, I think Java (VM and the byte code in particular, and to somewhat less extent the language) is still incredibly useful to the large-scale client side development. Especially so if a bit more can be added, such as a better interoperability with DOM and other HTML5 APIs. While JavaScript is getting better day by day, there are still many things Java does better.

Imagine being able to define a large application into modules, with interfaces to define boundaries, mature module systems, efficient delivery and caching mechanisms. Debugger support, multi-threading, runs outside the browser for unit testing, programming in Ruby or Groovy … you get the idea. Think of it as GWT except translation, if you may.

As an avid Java fan, here’s my plea to Oracle — I want this for my Christmas present this year.

Uncategorized

Internet Explorer only takes string in HTML5 postMessage

September 11th, 2012

Today I learned the hard way that our beloved Internet Explorer doesn’t handle HTML5 cross-domain messaging functionality (AKA postMessage) correctly.

I have done a reasonable amount of research before I invested a significant time into this, and so I knew it doesn’t postMessage across tabs and windows beforehand. This limitation was not a problem for me, because I was going to use this in IFRAME.

The problem that I discovered much too late is that IE only accepts a string as a message (see BSTR in a signature? That’s a Windows speak for passing UTF-16 strings.) This is despite the fact that the draft spec allowed any object and not just strings as early as Novemember 2010 .

And other sites like caniuse.com and various blog posts failed me too, as they failed to mention this limitation.

So I’m posting this here in the hope that this will prevent other people from wasting time.

AAARRRGGGGGGGHHHHH!!!!

Uncategorized , , ,

Jenkins User Conference New York: my take

June 13th, 2012

In one of those days, I’ll get a small enough computer whose battery won’t die in 45 minutes, but until then, my apologies for the belated Jenkins User Conference travel report.

So, as we tour around the world bringing Jenkins User Conference near you, the last stop was New York City.

I’ve kicked off the day by recaping the community activities in the past year, including Subversion 1.7 support, new UI effort, and several new extension points in the Matrix project. From my day job side, I’ve introduced BuildHive, a new addition to CloudBees DEV@cloud that builds your GitHub repositories in a few clicks, and a new version of Jenkins Enterprise by CloudBees with high-availability features.

There are two parallel tracks that went through the whole day, including talks from people who has been around in the community for a long time. Just to name a few, there’s Monty Taylor from HP discussing how OpenStack deploys Jenkins and Gerrit to ensure that the trunk never breaks, and Mike Rooney giving advices on running mission-critical Jenkins instances. Jesse Farinacci was also there, even though we couldn’t convince him to give a talk (and you can see his recap post in jenkins-ci.org.)

But aside from those who I already knew, there were many talks from whom I call “super Jenkins admins” — those who not only deploys Jenkins for their orgs, but push it to 11, by writing custom plugins, changing the visualization, writing peripheral programs that interact with Jenkins, etc.

For JUC NY, the super Jenkins admin award has to go to Jesse and David from AtTask. There are a number of awesome things they’ve done. First, they completely took the Jenkins instance to cloud, and not only do they provision build slaves from EC2, but also provision the entire Selenium Grid and its remote agents as well. Then they implemented a custom view plugin so that developers can see which stage in the pipeline their changes are at. They’ve also integrated Jenkins to their own product, AtTask, and automatically turns test failures into tickets (and it’s smart enough to reopen the old ones when a regression occurs!)

One thing I felt as I was listening to these incredible user story talks is that it’d be nice if we have means to capture the configuration of those instances and share those with others. Kind of like how you share cooking recipes. I’m not exactly sure how to achieve it, but it should include a list of plugins, some configurations, and it’d let you import those into your instance, so that you can replicate it, play with it, and modify it easily.

And last but not least, The talk from Noah about Jenkins REST API was also very good, in that it really highlighted a part of Jenkins that’s not widely used, and it gives everyone something that they can take back to their home.

I hope this post encourages you to come to future JUCs. Please submit papers, and register from the website. I’m really looking forward to the next one in Israel, and the one after that, Tokyo, has already signed up 700 people, so it’ll be a crazy event!

Uncategorized

Groovy SecureASTCustomizer is harmful

April 27th, 2012

I was looking at Groovy DSL slides from Guillaume Laforge when I noticed about SecureASTCustomizer, which led me to what appers to be the original introduction post from Cedric.

Being able to lock Groovy execution down would enable me to use Groovy in more places, so I did a bit of experiment. But I regrettably have to conclude that this feature is practically unusable. In fact I’d argue that it is actively harmful, as it gives a programmer a false comfort.

The fundamental problem is that Groovy is a dynamic language, yet SecureASTCustomizer works by looking at Groovy AST statically. So it’s very easy for Maloney, a malicious attacker, to bypass many of the checks. For example, Cedric’s post talks about how it can let you blacklist/whitelist classes that can be imported. Well, the actual goal of the programmer is to prevent the class from getting used, and not to get them imported. And sure enough, even if I white list the importable classes to java.lang.Math, Maloney can still do Math.class.forName('some.secret.class') to get a reference to a Class, and therefore render the import restrictions pointless.

Then I thought about disabling access to the getClass() method. But this doesn’t work well either because Groovy allows 5."class" and 5["class"] to access properties. To statically prevent this, you’d have to prohibit the array access and a string literal, but that doesn’t leave much of a language!

Many other checks offered by SecureASTCustomizer are equally useless. For example, there’s receiversClassesWhiteList that’s supposed to let you restrict the methods the script can invoke by whitelisting the declaring class of the method. But once again, this is a static check! Groovy compiler doesn’t work very hard to infer types, so much so that it can’t even guess that x=="foo" is a boolean type. Therefore, if you actually try using receiver whitelisting, pretty quickly you’ll discover that you either have to allow Object as a receiver (because Groovy assigns this to every expression when it couldn’t infer the type), which will basically renders the point of whitelisting moot as you can now invoke any method by simply casting the expression to Object.

If you go the other route and disallow Object as a receiver. That will reject almost all non-trivial scripts. Or I suppose you can prohibit a method call, but that doesn’t leave much of a language, does it.

Like I said, I think this is fundamentally a futile approach. You just can’t perform any meaningful static sandboxing on a dynamic language.

Instead, what I think is more fruitful is a dynamic checking. For example, what if the compile-time AST transformation intercepts every method call and property access? That is, transform z=x.y as z=checkedGet(x,"y"), transform x.y=5 into checkedSet(x,"y",5), and finally transform o.foo(a,b,c) into checkedCall(o,"foo",[a,b,c]). This does make execution a whole lot slower, but I can now perform meaningful checks. And unlike Java SecurityManager, this is a lot more friendly to libraries and web applications, who cannot take over the entire JVM.

I haven’t actually put together such an AST transformer, but this doesn’t look too hard.

What do people think?

Uncategorized , ,

COM4J updates

April 27th, 2012

It’s been a while, but I’ve posted a new version of COM4J. COM4J is a library that lets you talk to Windows COM components. Unlike similar libraries lika jacob, which makes you feel like you are working with reflection, COM4J is designed to work with type-safe annotated interfaces, which makes you feel like you are working with Java libraries. COM4J is also built on top of vtable invocation, not on IDispatch, so it can work with components without the dual interface support (boy those words bring back memories!)

I use this library in Jenkins, among other places, to provide a better native integration.

The major change in this version is that it finally has 64bit Java support. The original work was contributed in 2011, but I’ve never cut a release out of it officially. It contains a number of bug fixes, additional conversions support. The code is now on GitHub, and the website is moved to here.

Uncategorized , , ,

Debian and Maven, a crash of culture

March 16th, 2012

Tim O’Brien posted his frustration about the state of Java packaging in Debian. While I’m not affiliated with Debian nor Ubuntu, I wanted to post something in defense.

I completely understand where Tim is coming from. To the eyes of Java developers, the Java packaging in Debian looks completely Sisyphean. We got all the binaries and their dependencies captured in a machine readable form (aka POM). Can’t we just take them as-is, do a bit of metadata conversion, and make all those artifacts available to the Debian world so that we can just have a single package manager on Debian? If that’s your line of reasoning, you are in for a surprise, because Debian wouldn’t like that.

The reason they don’t do it is well summarized in the Debian Social Contract. It’s the equivalent of the U.S. Constitution for the Debian project — everything they do derive from this. Binary jars are bad for Debian because they don’t give the users the freedom to modify them and create derivative works. Debian is not just a means to let you conveniently install all the programs you need. It’s a pursuit of certain kinds of freedom.

In that sense, it’s somewhat like the “Free Software” movement. They both have some pretty strong guiding principles, and at times, for outsiders they look like they are “wasting” their efforts or being impractical. But the thing is, it’s those guiding principles that attract so many people to the effort, and that’s what keeps the project going and produce all the incredible good stuff that we use everyday. Criticizing them for their principles while you enjoy the benefits of the very same principles feel bit single-handed to me.

I think a better way forward is to write a little program that takes the source jar (which most jars in the Maven central should already have) and the POM, then generate a build script that simply compiles the source jar into the binary jar. The said program should also inspect the jar file to figure out any resource files, and treat them as source files. That way, we can machine-generate Debian source packages. Granted, not all source packages produced that way would pass the requirements of the Debian Freesoftware Guideline, but I bet substantial number of Maven artifacts are simple enough that this will be actually completely satisfactory. And then humans can concentrate on harder ones.

Anyone interested in giving that a shot?

Uncategorized , , ,

POTD: Package renamed ASM

March 3rd, 2012

Today’s project of the day is a package renamed ASM library.

I previously wrote about a problem in otherwise quite useful ObjectWeb ASM library. Namely, it breaks backward compatibility in such a way that badly breaks apps/libraries that use them. In that post, I wrote about two proposals to fix the pain point. One is to include debug information, which has been fixed starting 3.x. But the other package renaming hasn’t been addressed in the last 2 years.

This has been in the back of my head, but it never came high enough until recently I had another NoSuchMethodError caused by ASM3. One of the servlet containers shipped ASM3 and it broke Jenkins that bundles ASM2. Between this and ASM4 release for JavaSE 7, which will likely gain popularity over time, I finally decided to fix this problem once and for all, in a way that everyone else can reuse.

The solution, as explained in the original post, is to put each major ASM version in its unique package name. I pakage-renamed ASM2 in org.kohsuke.asm2, ASM3 in org.kohsuke.asm3, and ASM4 in org.kohsuke.asm4. The package name only contains the major version because I trust the ASM developers to maintain compatibility between minor releases (and I believe they’ve maintained this thus far.)

These artifacts are available in org.kohsuke:asm2:2.2.3, org.kohsuke:asm3:3.3.0, and org.kohsuke:asm4:4.0 — these are packaged renamed by jarjar and I tested them somewhat to make sure it’s not downright broken.

If library A depends on asm2 and library B depends on asm3, and someone else uses both A and B, everything will work fine because asm2 and asm3 are in the different pcakages. If A depends on one version of asm3 and B depends on another version of asm3, then the transitive dependency resolution will pick up the newer version and both will work (or you end up implicitly picking up one version over another, and you don’t enjoy the latest bug fixes, but at least it won’t die with LinkageError.)

When you search “asm3″ in Maven central, you see a large number of renamed ASM3 in various projects. Hopefully that madness will stop now!

The other interesting thing about this effort is that I’ve used Gradle to package rename them. Lately I’ve been using Gradle for publishing transformed artifacts like these to a Maven repository, and I like it a lot. But more about that in another post.

potd, Uncategorized

@Override and interface

January 27th, 2012

Jim Leary, my colleague at CloudBees, got me into digging into this.

The question is around putting the @Override annoation on a method that implements an interface method, like this:

public class Foo implements Runnable {
    @Override
    public void run() {}
}

As you can see in the javadoc, when @Override was originally introduced, such use was not allowed. javac 1.5 rejects this, too (I verified this in 1.5.0_22.)

Sun intended to change this in 1.6. Javac 1.6 indeed changed the behaviour to allow it (verified this in 1.6.0_26), but someone forgot to update the documentation, as you can see in the Java 6 API reference.

The interesting thing is, if you use Javac 1.6 with “-source 1.5″ and/or “-target 1.5″. In all the possible 3 combinations, the above code compiles. Is this a bug, or is this correct? The interesting thing is that the semantics of @Override is defined in the library, not in the Java language spec. So an argument can be made that this is as it should be — JLS, which governs the -source/-target switches, have nothing to do with this annotation. It’s akin to your code relying on newly introduced types in Java 6. If you compile them with Javac 1.6 with -source 1.5, it won’t raise an error.

But IDEs do seem to tie this with the language level. Jim said Eclipse, when set to language level 1.5, it will flag the above code as an error. I verified that IntelliJ does the same (but only in the editor, as the actual compilation happens via javac so the build will succeed.)

So the end result is ugly. If you open the project in your IDE, you see all these errors, but your build (nor test nor any actual execution, for that matter) will not catch this problem. Even if this was a bug in javac, I don’t see it getting “fixed” — the last thing you want is your security update relese to Java6 break all your builds.

I guess the right thing to do for projects (like Jenkins) is to try to avoid putting @Override on interfaces and as we discover them, remove them. So that people who open the source tree in IDE won’t see those false positive errors. This is a bummer because it’s actually useful to have @Override on interfaces (that’s why the behaviour was changed in 1.6 in the first place!) Does anyone know of a FindBugs rule or some refactoring tool to check this? Or should these be filed as bugs against IDEs? For enforcing something that’s not in JLS?

Uncategorized , ,

DNS outage with jenkins-ci.org

December 28th, 2011

As Tyler summarized it in this e-mail thread, currently there’s an DNS outage going on with jenkins-ci.org that makes all name resolutions fail.

The current ETA is right around the new year, but in the mean time, you can add our temporary DNS server into your /etc/resolv.conf via “nameserver 140.211.15.121″.

Once again our apologies for this outage.

Uncategorized