POTD: Confluence static cache generator plugin

I’m not sure about your Confluence, but my Confluence was dog slow. Page rendering regularly took a second or two, or even worse. That’s why I wrote this plugin.

This Confluence plugin generates static HTML files out of your Wiki pages. It happens every time when someone updates a page, post a comment, add a label, and so on. Anything that can affect the way the page looks in Confluence, and the cache gets regenerated.

Unlike Auto-export plugin, this plugin doesn’t try to apply a different theme, or help you make them look like non-Wiki sites, because the goal is the opposite. I want to make Confluence look faster, as opposed to generating website out of Confluence. To this end, HTML files generated by this plugin is a straight capture of what Confluence is rendering.

These generated files can be then served from frontend Apache reverse proxy. With the use of `mod_rewrite`, Apache will act as a smart caching proxy; it’ll serve static HTML files if they exist, and otherwise it’ll forward the requests to the backend Confluence.

The end result is that your visitors won’t even notice that your Confluence is running with cache. They see the same-old Confluence UI, and the same URLs will keep working. The only difference is that pages magically load a lot faster.

See the live instance to get the feel of how this cache works mostly transparently, and if you want to try this on your own Confluence, the code is here.

Jenkins User Conference New York: my take

In one of those days, I’ll get a small enough computer whose battery won’t die in 45 minutes, but until then, my apologies for the belated Jenkins User Conference travel report.

So, as we tour around the world bringing Jenkins User Conference near you, the last stop was New York City.

I’ve kicked off the day by recaping the community activities in the past year, including Subversion 1.7 support, new UI effort, and several new extension points in the Matrix project. From my day job side, I’ve introduced BuildHive, a new addition to CloudBees DEV@cloud that builds your GitHub repositories in a few clicks, and a new version of Jenkins Enterprise by CloudBees with high-availability features.

There are two parallel tracks that went through the whole day, including talks from people who has been around in the community for a long time. Just to name a few, there’s Monty Taylor from HP discussing how OpenStack deploys Jenkins and Gerrit to ensure that the trunk never breaks, and Mike Rooney giving advices on running mission-critical Jenkins instances. Jesse Farinacci was also there, even though we couldn’t convince him to give a talk (and you can see his recap post in jenkins-ci.org.)

But aside from those who I already knew, there were many talks from whom I call “super Jenkins admins” — those who not only deploys Jenkins for their orgs, but push it to 11, by writing custom plugins, changing the visualization, writing peripheral programs that interact with Jenkins, etc.

For JUC NY, the super Jenkins admin award has to go to Jesse and David from AtTask. There are a number of awesome things they’ve done. First, they completely took the Jenkins instance to cloud, and not only do they provision build slaves from EC2, but also provision the entire Selenium Grid and its remote agents as well. Then they implemented a custom view plugin so that developers can see which stage in the pipeline their changes are at. They’ve also integrated Jenkins to their own product, AtTask, and automatically turns test failures into tickets (and it’s smart enough to reopen the old ones when a regression occurs!)

One thing I felt as I was listening to these incredible user story talks is that it’d be nice if we have means to capture the configuration of those instances and share those with others. Kind of like how you share cooking recipes. I’m not exactly sure how to achieve it, but it should include a list of plugins, some configurations, and it’d let you import those into your instance, so that you can replicate it, play with it, and modify it easily.

And last but not least, The talk from Noah about Jenkins REST API was also very good, in that it really highlighted a part of Jenkins that’s not widely used, and it gives everyone something that they can take back to their home.

I hope this post encourages you to come to future JUCs. Please submit papers, and register from the website. I’m really looking forward to the next one in Israel, and the one after that, Tokyo, has already signed up 700 people, so it’ll be a crazy event!

Push changes directly into BuildHive, and never run tests again!

On top of pull requests auto-build, BuildHive now allows you to push changes directly in via ssh. I call this feature “validated merge”.

If you are an active developer of a repository, chances are that you don’t use pull requests to send in changes. You probably just push changes directly into your repository instead. But one of the common problems of doing this is that if your change breaks the build, you are going to discover it too late. With validated merge, this problem will never happen again, and this is how it works.

First, you add the buildhive user as a collaborator to your repository, so that it can push when the build is done. I didn’t feel comfortable doing this automatically, so you need to do it explicitly.

Second, login to BuildHive (just once is suffice, so that it learns your public keys you registered with GitHub), head to the job page in BuildHive (like this), and click “Git repository for validated merge” link from the left. Click the copy button to copy the “git remote add” command, paste that in your shell, and execute that on your local repository. This will add BuildHive as a remote repository.

Preparation is now complete. Go modify code like you normally do by creating commits, and when you are ready to push, push your changes to “jenkins” instead of pushing to “origin”. If you are on a branch, replace “master” with the branch name you are working on:

$ git push jenkins master

BuildHive will check out the current tip of the specified branch in GitHub, merge that with the changes you just submitted, then do the build and run tests. If the merge result successfully completes the build and tests, the result will be pushed to GitHub by BuildHive.

If you are unlucky and your changes weren’t as good as you thought, you’ll learn that it didn’t pass the builds. You then stash your work, come back to the broken changes, and rework those. You probably don’t remember which commit you pushed to BuildHive, so BuildHive provides a tag you can fetch into your workspace to get to it.

$ git stash
$ git fetch -n ssh://anonymous@buildhive.cloudbees.com/kohsuke2/sandbox-ant tag changes/45
$ git checkout changes/45
$ … make edits ...

You can add more commits to correct the problem, or you can even amend your commits — since your changes didn’t land in the repository, amending, rebasing, and so on is just fine. You can then re-push your changes and then go back to what you were working on before interrupted by a failure.

$ git push jenkins HEAD:master
$ git checkout master
$ git stash pop
… resume the work you were doing ...

If you’ve already headed home, your colleague can do this workflow for you.

Your time is precious. Don’t waste it by watching your laptop to complete tests. Instead, let BuildHive do it for you!

By the way, this is a feature already available in Jenkins Enterprise by CloudBees, so if you like the workflow but your source code isn’t in a public GitHub repository, you can do the same thing with any Git repository inside your firewall.