Archive

Posts Tagged ‘groovy’

Groovy folks, time to start agreeing

February 27th, 2015

I wrote about the drama unfolding in the Groovy project a month ago.

I left that topic for a while, but I was pleased to find out today that the question is no longer whether they need to move to a foundation, but rather which foundation it should be. There’s an email thread that has 188 messages and counting, going for more than 2 weeks, where the community is trying to figure out where to go.

I feel a bit of deja-vu, and I feel like I know exactly what’s going on. And no, I don’t mean the Jenkins drama, though we were in a relatable situation. Instead, I was thinking when I and my wife bought a house 5 years ago.

So there were two houses we really liked. The white house and the beige house. Like any informed buyer would do, my wife and I start disecting pros and cons. The white house felt a lot brighter and airy than the beige house, but the beige house has a bigger backyard. The white house is closer to a hospital, which might mean more noise. The beige house has a big tree nearby, and the gutter might fill up. List like that went on and on.

Now, the thing is, at a certain point, a list like this gets more confusing than helpful. I can add up all the pros and cons, but it doesn’t help me getting any closer to the decision making. In fact I’m no longer sure if the white house I wanted was really such a good idea. After all, it has no less than a dozen things listed under “cons”. I can see that my wife is getting just as confused as I am. The conversation starts to go in circles. I was lucky enough that our parents were living in the other side of the Pacific ocean, so we kept this conversation to ourselves. Otherwise, I’m sure it would have been even worse. They mean well, but sometimes too many opinions are more harmful than helpful.

In 1am in one of those long nights, I finally realized that agreeing on something, anything, is more important than figuring out the absolute best house out of two. So like every husband would do, I started trying to talk myself out of the white house I originally wanted, and speak about things I liked about the beige house that my wife originally wanted. I tried to downplay the concerns she had about the beige house.

Even though I started doing this consciously, the strange thing is that I started getting convinced by my own arguments that I didn’t fully believe in. Sure, the beige house doesn’t have the attic room that’s going to be my LEGO room, but I can get a brighter office and maybe I should keep my LEGO there so that I can occupy myself if meetings get boring. And in the end, we bought the beige house and we still live there mostly happily.

I feel like it’s time for Groovy guys to start doing this “let’s agree, whatever it is” dance. Judging from the conversations, I think they’ve figured out that they can live in any of these three foundations. The trick is not to get caught up on all the gory details, because there will be always something you don’t like. Yes, voting might be burdensome. Yes, losing @author tag might be annoying. Yes, infra migration would be painful. But you’ll be all right, and you’ll get used to it sooner than you think.

It’s time for people in the community to give the project leaders a blank check. I think we should be able to all trust them that they have the best interest of the project in mind.

And more importantly, it’s time for the project leaders to start converging. You guys need to sense where your consensus is heading to, and try to talk yourself into it. Try to create an echo chamber.

Engineers aren’t the best people to do this, but you guys really need to do this, because the clock is ticking. The difference between foundations is relatively small, but the difference between moving forward and procrastinating is huge. It’s a part of the leadership responsibility to form a consensus and then turn around and sell that to everyone, so that everyone feels better about what’s being done.

Remember, there’s really no wrong answer. Just different correct answers.

misc

Groovy project should have a clear governance structure

January 20th, 2015

I just came back from Tokyo to learn that the Groovy project is looking for a new home. Related posts from the project leaders here, here, and here. Hacker News commentary is here.

This news hit close to home for me for several reasons. For one, I like Groovy a lot myself, to the point that I have developed several projects around it (like this and this.) Two, the Jenkins project uses Groovy a lot in various places. I think Groovy has a number of things going for them, such as great IDE support (and the optional static typing boosts this), greater compatibility with Java, and an existing ecosystem. So my best wishes to them.

One of the things I realizing while reading the news is that I actually don’t know the governance structure of the project. Is the name “Groovy” trademarked? If so, who owns it? How about the domain name? How is the decision making done? Who becomes committers?

Questions like this are important at times like this, because we don’t know how much is owned by Pivotal, and that determines how much is up in the air.

For example, had Groovy been an Apache project, then we’d know the answers to all of the questions above, and this news really just boils down to who would be willing to hire the team and let them work on the project. Maybe they won’t find a single company that is willing to hire everyone and put them all full time on open-source Groovy — I bet they talked to prospects in private before going public, so at this point I assume the chance of this happening is pretty low. But I have no doubt they will each find a gainful employment that does involve some open-source Groovy work.

In contrast, if Groovy is a company-sponsored open-source project like Spring was, in that the company owns all the key assets and dictates the development process, then we have a lot bigger problem at hand, because there’s greater uncertainty. The project would have a bigger risk of fragmentation (for example if the team gets hired by different competing companies.) Perhaps license would change.

This is why I think the ownership of the project should be thought separately from the ownership of the developers (i.e. who pays the salary of the key contributers of the project.) When the latter changes, having the former sorted out considerably reduces the impact. And this comes from my own personal experience dealing with Hudson/Jenkins. This is one of the reasons why the Jenkins project has a governance structure laid out.

So I’d like to encourage the Groovy project to sit down and clarify the governance. This should be a part of their “find a new home” plan. Maybe they could even just join Apache Software Foundation since the license is compatible, maybe they could come over to SPI, where Jenkins is.

At this point, I’d be very happy with a less active Groovy project without a corporate sponsorship. But I wouldn’t like to see a governance-less Groovy project. I think they can avoid that.

Update: I’ve written a follow-up post on this topic

misc , ,

POTD: Application configuration via Guice binding + Groovy

March 1st, 2014

Often I write my applications with Guice. I also often want to make those applications configurable externally. For example I might inject username and password for that app to talk to another app, I might configure some timeout value, and so on. I make these configuration values available in Guice, so that I can access them wherever I need them. All of this is pretty common in many other places, I’d imagine.

Given that all I’m doing here is to pass configuration values from left to right, I thought it’d be nice if I can write configuration directly as a Guice module by using Guice binder EDSL. Then I won’t have to parse and translate these configuration any more.

And that became my project of the day.

This little library allows you to write Guice binding definitions in a text file:

timeout = 3
bind Payment named "customer" to VisaPayment

From your program, you use GroovyWiringModule to load this configuration file:

Module config = new GroovyWiringModule(new File("/etc/myapp.conf"));
Injector i = Guice.createInjector(
  Modules.override( ... my application's modules ...)
    .with(config))

The end result is that the above script gets translated into the following binding:

bind(int.class).annotatedWith(Names.named("timeout")).toInstance(3)
bind(Payment.class).annotatedWith(Names.named("customer")).to(VisaPayment.class)

Using Groovy as the host language for DSL has other benefits. If you are using system properties or environment variables to configure something, you are basically stuck with strings as the only representation of the configuration. With Groovy, I can create a relatively complex object and bind them, or even put some logic to further obtain values from elsewhere:

bind Payment toInstance new VisaPayment(
  cardNumber: "1234-5678-9012-3456",
  expiration: new Date(System.currentTimeInMillis()+TimeUnit.DAYS.toMillis(30),
  cvv: new URL("http://secret.server/cvv").text) 

With the functionality in Guice to override definitions in one module by another, I can also even override bindings defined in programs, for example to get more logging, add a filter, etc.

potd , , , ,

POTD: Groovy Sandbox

April 29th, 2012

I posted the other day about Groovy’s SecureASTCustomizer and how it is harmful. In the end of that post, I suggested that doing the check dynamically would work better.

So in this installaition of “Project Of The Day”, I went ahead and implemented it. The result is Groovy sandbox.

My main idea is to confine the sandboxed script into its local object graph. The script should be allowed to mutate this graph all it wants, but it’ll be only allowed to include objects of known whitelisted “safe” types (such as String, List, Date etc.), and a few known safe instances, which acts as a bridge between the sandbox and the rest of the world. These bridge objects would have to be written carefully.

To use this, you have to add SandboxTransformer to your CompilerConfiguration first:

def cc = new CompilerConfiguration()
cc.addCompilationCustomizers(new SecureTransformer())
def binding = new Binding();
binding.robot = robot = new Robot();
def sh = new GroovyShell(binding,cc)

And now any script compiled via the resulting shell object will be sandboxed.

When a sandboxed script executes, all of the following operations are intercepted.

  • static/instance method invocation foo.bar(....)
  • object allocation new Foo(...)
  • property access and assignment zot=foo.bar / foo.bar=zot
  • attributes access and assignment zot=foo.@bar / foo.@bar=zot
  • array access and assignment zot=foo[bar] / foo[bar]=zot

To examine those calls and reject some of them, create your own implementation of GroovyInterceptor and registers it to the thread before you start executing the script:

def sandbox = new RobotSandbox()
sandbox.register()
try {
    sh.evaluate("robot.leftArm.move()")  // this is allowed to complete
    sh.evaluate("robot.selfDestruct()")  // no!
} finally {
    sandbox.unregister()
}

See the robot example for a complete example.

Now let’s see if I can get some feedback from real Groovy experts, and see if they’d be willing to take this into Groovy itself…

potd , ,

Groovy SecureASTCustomizer is harmful

April 27th, 2012

I was looking at Groovy DSL slides from Guillaume Laforge when I noticed about SecureASTCustomizer, which led me to what appers to be the original introduction post from Cedric.

Being able to lock Groovy execution down would enable me to use Groovy in more places, so I did a bit of experiment. But I regrettably have to conclude that this feature is practically unusable. In fact I’d argue that it is actively harmful, as it gives a programmer a false comfort.

The fundamental problem is that Groovy is a dynamic language, yet SecureASTCustomizer works by looking at Groovy AST statically. So it’s very easy for Maloney, a malicious attacker, to bypass many of the checks. For example, Cedric’s post talks about how it can let you blacklist/whitelist classes that can be imported. Well, the actual goal of the programmer is to prevent the class from getting used, and not to get them imported. And sure enough, even if I white list the importable classes to java.lang.Math, Maloney can still do Math.class.forName('some.secret.class') to get a reference to a Class, and therefore render the import restrictions pointless.

Then I thought about disabling access to the getClass() method. But this doesn’t work well either because Groovy allows 5."class" and 5["class"] to access properties. To statically prevent this, you’d have to prohibit the array access and a string literal, but that doesn’t leave much of a language!

Many other checks offered by SecureASTCustomizer are equally useless. For example, there’s receiversClassesWhiteList that’s supposed to let you restrict the methods the script can invoke by whitelisting the declaring class of the method. But once again, this is a static check! Groovy compiler doesn’t work very hard to infer types, so much so that it can’t even guess that x=="foo" is a boolean type. Therefore, if you actually try using receiver whitelisting, pretty quickly you’ll discover that you either have to allow Object as a receiver (because Groovy assigns this to every expression when it couldn’t infer the type), which will basically renders the point of whitelisting moot as you can now invoke any method by simply casting the expression to Object.

If you go the other route and disallow Object as a receiver. That will reject almost all non-trivial scripts. Or I suppose you can prohibit a method call, but that doesn’t leave much of a language, does it.

Like I said, I think this is fundamentally a futile approach. You just can’t perform any meaningful static sandboxing on a dynamic language.

Instead, what I think is more fruitful is a dynamic checking. For example, what if the compile-time AST transformation intercepts every method call and property access? That is, transform z=x.y as z=checkedGet(x,"y"), transform x.y=5 into checkedSet(x,"y",5), and finally transform o.foo(a,b,c) into checkedCall(o,"foo",[a,b,c]). This does make execution a whole lot slower, but I can now perform meaningful checks. And unlike Java SecurityManager, this is a lot more friendly to libraries and web applications, who cannot take over the entire JVM.

I haven’t actually put together such an AST transformer, but this doesn’t look too hard.

What do people think?

Uncategorized , ,