Home > Uncategorized > Debian and Maven, a crash of culture

Debian and Maven, a crash of culture

March 16th, 2012

Tim O’Brien posted his frustration about the state of Java packaging in Debian. While I’m not affiliated with Debian nor Ubuntu, I wanted to post something in defense.

I completely understand where Tim is coming from. To the eyes of Java developers, the Java packaging in Debian looks completely Sisyphean. We got all the binaries and their dependencies captured in a machine readable form (aka POM). Can’t we just take them as-is, do a bit of metadata conversion, and make all those artifacts available to the Debian world so that we can just have a single package manager on Debian? If that’s your line of reasoning, you are in for a surprise, because Debian wouldn’t like that.

The reason they don’t do it is well summarized in the Debian Social Contract. It’s the equivalent of the U.S. Constitution for the Debian project — everything they do derive from this. Binary jars are bad for Debian because they don’t give the users the freedom to modify them and create derivative works. Debian is not just a means to let you conveniently install all the programs you need. It’s a pursuit of certain kinds of freedom.

In that sense, it’s somewhat like the “Free Software” movement. They both have some pretty strong guiding principles, and at times, for outsiders they look like they are “wasting” their efforts or being impractical. But the thing is, it’s those guiding principles that attract so many people to the effort, and that’s what keeps the project going and produce all the incredible good stuff that we use everyday. Criticizing them for their principles while you enjoy the benefits of the very same principles feel bit single-handed to me.

I think a better way forward is to write a little program that takes the source jar (which most jars in the Maven central should already have) and the POM, then generate a build script that simply compiles the source jar into the binary jar. The said program should also inspect the jar file to figure out any resource files, and treat them as source files. That way, we can machine-generate Debian source packages. Granted, not all source packages produced that way would pass the requirements of the Debian Freesoftware Guideline, but I bet substantial number of Maven artifacts are simple enough that this will be actually completely satisfactory. And then humans can concentrate on harder ones.

Anyone interested in giving that a shot?

Uncategorized , , ,

  1. March 16th, 2012 at 11:05 | #1

    This sounds really noble, but I don’t think you see how that group works. They already have these scripts to try to divine the build from a source jar. It’s pretty crazy over there. If we can’t render judgements on open source communities just because they produce “incredible good stuff we use every day”, what’s the use of even having an opinion?

  2. March 16th, 2012 at 22:09 | #2

    Thank you for this explanation and defense. I have two resources to add:

    - A list of recent opportunities to get infected with malware by trusting unsigned binaries:
    http://www.koch.ro/blog/index.php?/archives/153-On-distributing-binaries.html

    - Some merits of compiling from source in the Scala/Java world
    http://blog.ometer.com/2012/01/24/the-java-ecosystem-and-scala-abi-versioning/

  3. March 17th, 2012 at 08:31 | #3

    I think its fine to have an opinion but listening is also an important skill. The discipline of building from source uncovers all kinds of problems and lets you do broad optimization that isn’t otherwise possible. Java focuses on a cross-platform bytecode, so this may seem less important. What if someone came up with a way to get 20% more performance out of java code on 64-bit AMD but it required you to rebuild JARs from source? If you have source as part of the package and all packages build from source, this is trivial. If you distribute binaries then you may have packages in your distribution that *no one* has the source for. How sure is the maven community that some of the source that produced certain binaries was lost because of bad backup policies? Do you really want those kinds of packages on your computer?

  4. Anderas Jackson
    March 17th, 2012 at 13:58 | #4

    I should read more about this, but it sounds like all complains from Ruby group about Debians packing of Ruby and RoR.
    They, Ruby group, didn’t understand the goals of Debian and how the Ruby way of working was in total collision with those.

    And Tim, you are not the only one with an opinion. So other are also allowed to give you some critics, are they not? That will not remove your right to complain, even if others think that your critics are not fair.

  5. March 18th, 2012 at 01:06 | #5

    Hi,
    I think that this should be available on a repository (nexus or some others) thus enabling the generation of deb binaries and source files. Or maybe a simple maven plugin for libs, but for programs something more is required.
    But I think also there is some misunderstanding on how java/maven works since they want to have only one version of a lib (if you look at Ubuntu rules on the wiki they explicitely say that the code should work even if the lib version is incorrect).

  6. March 18th, 2012 at 23:14 | #6

    Clear up my confusion please… What exactly is the beef that debian has with maven?

    I can’t see that it is because the source code is not available, because for the majority of libraries in maven repositories, it is, so users can change the source code and recompile.
    So is it because the binaries don’t get compiled for a particular platform? That goes against the grain of java itself with the write once run anywhere axiom.
    Or maybe it is just a packaging thing?

  7. RIck Thomas
    March 20th, 2012 at 11:46 | #7

    Thanks for the clear explanation. As a regular Debian user and contributor (though not a “developer”) I understood this already, and accepted it as a condition of use. Tim seems to feel that he can take without giving back, and criticize when the things he takes aren’t exactly what he likes.

    The Debian folks have a standard response for this kind of attitude: “Patches are welcome.” If you don’t like it, you have the power to fix it. If you fix it, share the fixes.

  8. March 21st, 2012 at 00:00 | #8

    @Michael Wiles
    The “beef” is that you cannot build binaries from the source code. Yes, Maven repositories can and do often have source jars, but those by themselves aren’t buildable, and in Maven ecosystem we don’t rebuild them. This isn’t about freedom to compile for a particular platform. It’s about the freedom to modify code and then build binaries.

  9. March 21st, 2012 at 00:09 | #9

    @RIck Thomas
    Being a Maven user, I also see the other side as well. Yes, there is a risk that I as a Maven user is running when I download binaries and use them instead of rebuilding it from sources. But hey, it’s convenient as hell, it works for me, and I take all kinds of risks everyday anyway (like driving a car.)

    When I decide to use a library, there’s a certain trust I put to the developers of the said library. Extending that trust to assume they don’t put malicious code into the binaries they publish in Maven repositories doesn’t sound all that unreasonable (and thanks to PGP we can cryptographically verify that there’s no tampering in the middle.)

    I’m not trying to pass the judgement here. I’m just saying this is a crash of two cultures, and both cultures have a pretty “reasonable” logic.

  10. March 26th, 2012 at 15:41 | #10

    As the author of ‘el loco’ Maven repo spec for Debian, I can understand the frustrations of Tim. The Maven repository as revisited by Debian is a compromise between the need to stay compatible with upstream Maven, and deal with the constraints coming from Debian (build from source, try to limit the number of versions for a library to avoid being lost in packages, allow smooth upgrades from a version to another so that a library shared by different software won’t break anything when upgrading). This result into something a bit unatural for Maven users, but it’s there for a good reason, and it’s not meant to be used by casual programmers!

    But it’s a stepping stone into building more complex Java software into Debian, and benefit from the awsome channel of distribution provided by Debian, Ubuntu, Mint and so many others into end-user computers, servers and clouds. Jenkins benefits from this distribution channel, so does Tomcat, Eclipse and so many other Java software painfully built from source.

    The build from source philosophy may seem a burden, but didn’t you loose a couple of small but useful projects during the awful migration of dev.java.net? I did, and I wish I had a backup of those sources.

    Sysadmins are the main users and drivers of Debian, and they need more certainty when installing a new piece of software on their systems. The first thing a good sysadmin will ask is where does this software comes from? Debian? Known and trusted. Github? No way ;-)

    I hope this clarify some concerns, and shows you a bit of the value of the work done in Debian.

    Cheers,
    Ludovic

  11. March 26th, 2012 at 15:45 | #11

    I forgot to add, for those interested. There is a tool called mh_make, available from the package maven-debian-helper, which tries to automate most of the work for transforming the sources of a Maven project into a Debian package respecting all Debian policies. Still work in progress, but getting useable and useful.

  12. April 10th, 2012 at 19:51 | #12

    Here is a page that explains some of the things the Debian folks are trying to achieve: http://wiki.debian.org/Java/MavenBuilder

  1. No trackbacks yet.