Art of asking questions

A part of my work is a journalistic work. I go out, I grok how & what software people are building, and that knowledge informs what we do next. This is primarily done by asking questions and listening to the answers. So I think a lot about how to ask questions.

One of the first things I noticed is that if you are trying to find out something, it’s never a good idea to ask a direct question, such as, “does X matter to you?” Because for almost any meaningful X, say cost, culture, people, process, … the answer will be “yes,” but that’s useless knowledge. What you really want to ask is how people trade off two good things X and Y. The world is full of good things we don’t get around doing it.

Another problem with these piecemeal speed dating questions is that every time you come up with a new question, you have to go back to them and ask. My knowing what color you like and what food you like doesn’t really help me infer what car you want to drive.

Instead, I strive to understand the picture they are seeing. What do they perceive as the reality around them? What are they trying to change in there and what do they see as constraints? Why? Tell them somebody else’s story in a similar situation and ask them how those are different? Ask them to draw diagrams, or draw a diagram yourself and ask them to correct your misunderstanding.

When I’m successful in doing that, I can put myself in their shoes. I can think like them, and I can infer what they do for why. I can retrace their decisions. I pitch some of those back to them, and see if I got that right. If there are discrepancies, that’s a sign that I grossed something over. Once I build such a mental model, it’s very powerful. I can answer the speed dating questions myself based on it.

This takes time. I usually ask for 2 hours to conduct these interviews. I then take several more hours to process the raw notes into a readable material. It also only makes sense in a small circle setting, ideally 1:1. IMHO a group discussion is useless if you are trying to really understand anything. Some people dominates the air time and others disengage. People use the same word to mean different things. They latch on to details that are irrelevant. I’m sure you know what I mean.

Yet I see a lot of people asking speed dating questions, in a group setting. And that’s been puzzling to me. When you put 10 people in a room and ask questions like “is testing important?” what kind of knowledge can you possibly gain out of it? What am I missing?

On technology leadership

What does it mean to be a technology leader? This has been and still is a perpetual question for me. In this post, I’d like to reflect on what I learned about technology leadership in my experience as the creator of Jenkins and as the CTO of CloudBees.

Leading by Code

When a software engineer rises in prominence through the popularity of their work, the first phase of the leadership is what I call “leading by code.” People around you notice that you are doing some great work, and they start paying more attention to you. By this point, you own a lot of code, meaning you’ve probably written quite a bit of code of the said software yourself, including the heart of the design and the architecture. Owning code gives you upper hand in many ways, which you can use to influence others:

  • You probably are the most knowledgeable person for many parts of the system, so whenever somebody tries to build something anywhere, they want your input, approvals, or maybe even some matching change from you.
  • Your code, coding style, and how you produce it all become the baseline for everyone. In trying to do “monkey see monkey do,” they seek your input and approvals.

In essence, you lead by being “a better doer.” I remember that this felt great. Engineers close to me did not need a title change to know the influence I was radiating, but now my boss, my distant friends, and people who are looking at my LinkedIn resume knows that I’m a leader! I also obviously believed that my way of doing something is great, and I was helping others do it better. I wasn’t really changing anything I was doing before – I kept writing code, just more often together with other people. When I wanted something to be done in a certain way, it was usually followed. Everyone wanted my input. What’s not to like!?!?

Limit of Leading by Code

I think “leading by code” comes naturally to everyone. I’ve seen many senior engineers who perpetually stayed in this comfortable mode. But today I believe it breaks down at a certain size of a team. I now think, perhaps selfishly, trekking beyond this comfort zone is what differentiates a technology leader from a better software engineer.

Why does “leading by code” break down? Simply put, you become a bottleneck.

  • Bandwidth: if everyone needs your attention and focus to get their job done, then you run out of time. When people start doing things increasingly on their own as a result, you feel threatened, because you feel your control is slipping away and other people are developing deep expertise in various pockets.
  • Creativity: when you influence others by basically controlling what & how they do, either directly by telling so, or indirectly by controlling a tribal custom, it naturally limits other people’s creativity. This is a hard one to see when you are in the middle of it, because there’s always some room of creativity that you can point to.

That is me in 2019 speaking. I didn’t have a perspective back then to think in these terms. So I’ve probably stayed in this mode of leadership for too long, feeling increasingly unhappy and disappointed at myself. There were a few signs that this leadership style was hitting a limit.

  • At one point, I wanted to design and write an interface between two subsystems myself, because “owning code” was how I was exercising the influence. People implementing the two subsystems weren’t happy at all that I was lagging behind, producing something different from what they wanted. It didn’t help me either because I lost some credibility in their eyes, rather than gaining influence!
  • I was in a design fight with somebody under me, and I couldn’t resolve it in an amicable manner and ended up just overriding him. I remember him saying “I’ll reserve the right to tell you I-told-you-so.” I had some reasons for doing that, but really that’s beside the point. He got demotivated, and it created “my way or your way” framing. He’s also now rooting for you to be wrong, and sooner or later, you WILL be wrong!

So I started exploring other mode of technology leadership.

Launching a New Project

One of the things that worked was to focus on launching a new project/product/effort. The idea is that you basically get embedded into a team for a while, then transition out when the time is right. I’ve done this several times, and I think it suited me well for several reasons:

  • I could “lead by code” with all its benefits without much of its downsides because new things start small. I could steer a team toward a general direction without making people feel like you are telling them what they do.
  • New things benefit from an executive sponsor who communicates progress upward, keeps unwanted disruptions at the bay, builds bridges to where necessary, brings input, context, and perspectives from other parts of the org.
  • When a new team is formed to take on a new thing, which I suspect is a norm in a startup, I can help the forming and norming phases. I bring in “how things are done around here” naturally thanks to the title, and speed up initial decision making. Just don’t forget that you will be wrong on some of those decisions, and you got to let the team take over and fix some of those. That used to feel embarrassing, but now I take it as a sign of a maturity of the team.
  • I suspect one of the things that keep the CTO role influential is being on the side of change, as opposed to being the defender of the reality on the ground. Being seen as a part of new projects is helpful with this regard.
  • And new things are fun, so it kept me engaged!

Storytelling

As the org grew, the number of people I needed to influence got larger, and I found myself leaning more and more toward a meaty, structured write-up, as opposed to email dialogs, or 1:1s with key individuals. This blog post itself is a good example of such a write-up, albeit too long.

A write-up can start with a recap of context, or cross references to earlier write-ups, which help larger number of people to “get it.” Writing down helps you organize your thoughts more clearly and think more thoroughly. Your message also won’t get lost in Chinese whisper game, when people forward your message to others. It takes more time to compose one, but the wider reach makes it worthwhile.

CEOs and managers of big orgs do this all the time, but I think technology leaders should do this more, too. And you don’t need to use the “unicorns and rainbows” voice that they use, either!

I started consciously doing this more and more after getting hooked to great storytelling podcasts like This American Life and Radiolab. Those podcasts got me thinking a lot about how to tell a story better. When it works, it makes your write-up more engaging, and it frames how people think about a certain thing. When you succeed in that, there’s no need to sell people a solution. I’ve sought some help from people who studied journalism, and I’m seriously wondering if there’s more training/class available in this area.

Aside from one-off write-ups, I’ve run two series of write-ups that have certain themes, and both had great tractions. One was a series of “portraits” of people who we are building our products for, and the other is things I’m noticing about what’s going on in the industry, such as event reports.

I’ve done most of this on Google Doc, and what I loved is that people can comment on my message to add more colors, which makes it a whole lot more valuable. OTOH, Google Doc sucked at helping people find your write-ups, and that took a lot of value away from it. If Confluence was more loveable experience, I would have switched.

1:1s with CTO

At around ~100 people, and as I developed a “storytelling” broadcast channels, my 1:1 interaction with engineers started to change in nature. It used to be “outbound” where I tried to influence key senior people. It now became more “inbound” where I learn great ideas, interesting efforts, and/or their challenges from people. People were much more willing to talk about that in 1:1 and all I needed to do was to prime the pump and listen attentively.

So I started to encourage them, help them get more time working on them, and so on. Basically, find a pocket of excellence and try to lend my credibility to grow them.

It’s much easier to make something happen when there’s already somebody who’s thinking about it and passionate about it, even considering the downside that is “it’s not exactly how I wanted it.” I’d also like to think it creates a workplace that engineers want to be in.

I’m still trying to figure out how best to do this. When you skip reporting chains and go directly to ICs, it can be disruptive. You can’t create a change without creating any friction, but you want to minimize that. Also, not every ground-up effort pans out, and when they fail for one reason or another, it can be really disappointing for the person I encouraged. Expectation setting can be tricky, and so much depends on individual context and the personality. When an effort does pan out, the next stage of the project needs more serious organizational commitment, and I often struggled to bridge that smoothly.

Toward the end, I started thinking about creating a rotating position to do these “incubation” projects by pulling engineers out of their regular positions and time-boxing projects. This felt like a promising idea – it makes the whole thing a repeatable program that other people would recognize better, and hopefully that encourage more people to bring ideas to me and help with the “next stage” problem.

Architecture Forum

I’ve seen this done in many places, but I’m not sure if I cracked the nut. I’d love to hear from people who has done it successfully. The idea is that you have many teams with defined territories, and you drive the alignment/collaboration between them through meetings where people from those teams come, present, and talk to each other.

  • I thought the way to make this valuable is to ask them questions. What about observability? Have you thought about this failure mode that happens all the time? Help them tap into organizational memories. Bring concerns and realities from the field. But I felt one-hour session is too short for that, and there’s always somebody from other teams who ends up in the “let me design it for you” mode who speaks too much.
  • When I invited one person from every team to be on this forum, some people thought “hey how come Alice is invited but not me?” and that made them unhappy.
  • I suspect many engineers just don’t have the mindset of “there’s always something to be learned from new perspectives” baked into them, and those people don’t feel a real value from this setup. When enough people do not feel value, it’s too easy for this to become a bureaucratic formality.

In retrospect, I think a better way to do this is eng-org wide continuous education. I could have identified people who can speak to those concerns, perspectives, and context, and perhaps I could have a dialog with them and capture that into a presentation, a document, a podcast, or something. That way I can amplify a voice of somebody while still creating a crafted message that I want the org to hear, and I can influence everyone, not just tech leads. And so long as it sticks to somebody in a team and they speak up, the team will be doing the right thing.

Conclusions

This post is really more of a progress report of mine than a guide book for others. Your mileage will vary.

That said, if there’s any take away for others, I think it’s that the PDCA cycle works. That is, you try something, notice that it seems to be working, so you build/adjust a mental model of why that works, then that helps you plot the next move. What started as a random search and blind shots gradually become more systematic hypothesis testing over time. You never really find answers (at least I didn’t find one), but you do find questions that are worth asking.

It was uncomfortable at the beginning, but once I felt that the cycle was going, I felt happier because I was making a progress. Then the rest became easy.

I hope you’ll find some of this useful, and I’d love to hear from people what worked for them and didn’t.

My personal journey around diversity

Today, in the software development industry, the issue of gender, racial, and other diversities is widely accepted. But I also think this is still largely understood as “social value,” as opposed to “business value.” As a result, the awareness is not leading to enough actions, because it loses out in the crucial fight over priority. Like all other good things, it never get acted on.

This is a story of my personal experience that helped me develop new perspectives on this issue, to more clearly see why it’s in your business interest to tackle this.

Where I started

I’m a left-leaning man born to Japanese parents, who spent equal time between the U.S. and Japan. In other words, I’m generally on the side of increasing diversity in any given team. I’m also a father to a daughter. So I’ve always felt very sympathetic to the gender diversity issue. it was obviously a good value to uphold.

I was also very proud of the geographical and cultural diversity of the Jenkins community and CloudBees. I remember in one dinner we had, I looked around the table and realized that this group of 8 people represented 6 countries and 6 native languages, and I loved it.

I’ve been feeling like I was a supporter of the cause, a part of the solution.

What made me pause

Then I hired Tracy Miranda as a new community manager to my team at CloudBees. She was going to help us grow the Jenkins community. She was very passionate about the issue of diversity, she led that effort in the Eclipse foundation. So it was no surprise that in one of the planning meetings, she proposed that our team prioritize tackling the gender diversity problem.

A surprise was my own reaction. I found myself not really buying into it. The question wasn’t whether that is a good thing to do. It obviously is. The question was whether it’s better than all the other good things we can do.

I didn’t feel like she gave me that rationale, I couldn’t answer this myself, and I felt other people in the team had the same question.

That’s when I realized maybe I wasn’t a part of the solution, I was a part of the problem, a part of the system that created inertia.

What I run into

That realization wasn’t a pleasant one to face, but it also gave me an important question — what is the business value of the diversity? Fortunately, within a relatively short period of time, I randomly came across two great stories that helped me explore this question.

  • The story of Arlan Hamilton: This series of podcasts tells a story of a gay black woman trying to start a venture capital. She makes the observation that startup founders are not reflecting the general population balance, and therefore her core investment thesis is that there’s a untapped business opportunity in investing in minority founders. It was interesting that she doesn’t see her work as a social responsibility or charity work. It’s simply a space the competition was failing to dominate, an untapped field, and she was going to win that space.
  • Curb cuts: This 99% invisible episode talks about ‘curb cut,’ which is a slope from a pedestrian walk to a road. This was originally done to make roads more accessible to wheelchairs, but when it happened, it turned out that everybody benefited. Parents with strollers, people with suitcases, etc. What I got thinking is that when we make a community accessible to one under-represented group, the effort will have a ripple effect to other under-represented groups. This is a very common thing in software engineering. When we solve one use case, we often solve it in a bit more generalized way, and that helps other use cases.

What I learned

From these stories, I started to see the issue of diversity differently.

It’s like opening a new bar and in a new town. You build a bar that you like, you hang the “open” sign, and you wait for people to show up. Some people wonder in, the place gets busier, your regulars give you feedback, and you improve your bar, which brings more customers. Things are going great.

But at some point, this kind of growth will plateau, because you are self-selecting your customer base. At some point, to ignite the next growth, you need to do something else.

What you do is to go out on the street, see people who are not coming to your bar today, then think about what it takes to make them your customers. You might notice that there are a lot of Hispanic people walking around in your neighborhood. Maybe you’ll prepare a menu in Spanish, or put pictures in the menu. And you repeat this cycle to win more customers.

If you think about it this way, it’s obvious. Don’t sit and wait. Reach out. In marketing, for example, people do this all the time. But among software companies or open-source projects, too many people are still in the “open a bar and hope people show up” mode, and not realizing the lost business opportunities.

And that’s because this is also hard. It’s hard to think about and understand people who are different, who you don’t see.

How I connected dots

And on the point of how hard it is to think about and understand people you don’t see, I have a lot of personal experience.

I grew up in Japan, so I’m well connected in the software development scene over there. I know countless faces of great software engineers, team leaders, and open-source people over there. It’s the number 3 economy in the world, after all! But here in the west, they are just not on the radar. The open-source people I interact in the U.S. and Europe simply don’t know that these people exist, and so they don’t know what they are missing. That’s something I felt personally passionate about, and I realized the structure of this is exactly the same with the gender diversity.

I had another “aha” moment of this exact same structure from another angle, when I visited China and saw its thriving tech scene. I realized many obstacles we unknowingly set up in the Jenkins community. Our mailing lists are on Google Groups. Too bad, Chinese firewall blocks them. Our official twitter feed was similarly blocked. If we hope to reach Chinese users, we need to be on their social network that none of us knew. Something that I was completely unaware of, that became totally obvious once I met them and talked to them.

Conclusion

As the software engineering community, we need more good engineers to join our teams, our communities. And a rational way to do so is to build a new muscle to seek out untapped engineers and win them. Find, talk, and understand a group of people different from you. Accommodate and win them. As with everything else, if you’ve never done it before, doing it for the first time is hard. But you will get a hang of it pretty quickly.

I think any group of people would do. When you solve it for one group, you are already halfway there solving it for other groups.

Gender just happens to be a relatively “obvious” one to tackle first. It just acts as a concrete goal of the otherwise abstract idea of “reaching out.” One, the total addressable market (TAM) is pretty big — it’ll basically double your current TAM. There’s no firewall, no time zone difference, no language barrier. It’s also a very visible group that everyone can see, which makes reaching out much easier. On top of that, it’s a well trodden path with a lot of collective experience to tap from.

So that’s where I stand in 2019. Hopefully my journey is useful to others going through the same journey. Looking forward to hearing from people who have thought more about this.

Groovy folks, time to start agreeing

I wrote about the drama unfolding in the Groovy project a month ago.

I left that topic for a while, but I was pleased to find out today that the question is no longer whether they need to move to a foundation, but rather which foundation it should be. There’s an email thread that has 188 messages and counting, going for more than 2 weeks, where the community is trying to figure out where to go.

I feel a bit of deja-vu, and I feel like I know exactly what’s going on. And no, I don’t mean the Jenkins drama, though we were in a relatable situation. Instead, I was thinking when I and my wife bought a house 5 years ago.

So there were two houses we really liked. The white house and the beige house. Like any informed buyer would do, my wife and I start disecting pros and cons. The white house felt a lot brighter and airy than the beige house, but the beige house has a bigger backyard. The white house is closer to a hospital, which might mean more noise. The beige house has a big tree nearby, and the gutter might fill up. List like that went on and on.

Now, the thing is, at a certain point, a list like this gets more confusing than helpful. I can add up all the pros and cons, but it doesn’t help me getting any closer to the decision making. In fact I’m no longer sure if the white house I wanted was really such a good idea. After all, it has no less than a dozen things listed under “cons”. I can see that my wife is getting just as confused as I am. The conversation starts to go in circles. I was lucky enough that our parents were living in the other side of the Pacific ocean, so we kept this conversation to ourselves. Otherwise, I’m sure it would have been even worse. They mean well, but sometimes too many opinions are more harmful than helpful.

In 1am in one of those long nights, I finally realized that agreeing on something, anything, is more important than figuring out the absolute best house out of two. So like every husband would do, I started trying to talk myself out of the white house I originally wanted, and speak about things I liked about the beige house that my wife originally wanted. I tried to downplay the concerns she had about the beige house.

Even though I started doing this consciously, the strange thing is that I started getting convinced by my own arguments that I didn’t fully believe in. Sure, the beige house doesn’t have the attic room that’s going to be my LEGO room, but I can get a brighter office and maybe I should keep my LEGO there so that I can occupy myself if meetings get boring. And in the end, we bought the beige house and we still live there mostly happily.

I feel like it’s time for Groovy guys to start doing this “let’s agree, whatever it is” dance. Judging from the conversations, I think they’ve figured out that they can live in any of these three foundations. The trick is not to get caught up on all the gory details, because there will be always something you don’t like. Yes, voting might be burdensome. Yes, losing @author tag might be annoying. Yes, infra migration would be painful. But you’ll be all right, and you’ll get used to it sooner than you think.

It’s time for people in the community to give the project leaders a blank check. I think we should be able to all trust them that they have the best interest of the project in mind.

And more importantly, it’s time for the project leaders to start converging. You guys need to sense where your consensus is heading to, and try to talk yourself into it. Try to create an echo chamber.

Engineers aren’t the best people to do this, but you guys really need to do this, because the clock is ticking. The difference between foundations is relatively small, but the difference between moving forward and procrastinating is huge. It’s a part of the leadership responsibility to form a consensus and then turn around and sell that to everyone, so that everyone feels better about what’s being done.

Remember, there’s really no wrong answer. Just different correct answers.

Groovy project should have a clear governance structure

I just came back from Tokyo to learn that the Groovy project is looking for a new home. Related posts from the project leaders here, here, and here. Hacker News commentary is here.

This news hit close to home for me for several reasons. For one, I like Groovy a lot myself, to the point that I have developed several projects around it (like this and this.) Two, the Jenkins project uses Groovy a lot in various places. I think Groovy has a number of things going for them, such as great IDE support (and the optional static typing boosts this), greater compatibility with Java, and an existing ecosystem. So my best wishes to them.

One of the things I realizing while reading the news is that I actually don’t know the governance structure of the project. Is the name “Groovy” trademarked? If so, who owns it? How about the domain name? How is the decision making done? Who becomes committers?

Questions like this are important at times like this, because we don’t know how much is owned by Pivotal, and that determines how much is up in the air.

For example, had Groovy been an Apache project, then we’d know the answers to all of the questions above, and this news really just boils down to who would be willing to hire the team and let them work on the project. Maybe they won’t find a single company that is willing to hire everyone and put them all full time on open-source Groovy — I bet they talked to prospects in private before going public, so at this point I assume the chance of this happening is pretty low. But I have no doubt they will each find a gainful employment that does involve some open-source Groovy work.

In contrast, if Groovy is a company-sponsored open-source project like Spring was, in that the company owns all the key assets and dictates the development process, then we have a lot bigger problem at hand, because there’s greater uncertainty. The project would have a bigger risk of fragmentation (for example if the team gets hired by different competing companies.) Perhaps license would change.

This is why I think the ownership of the project should be thought separately from the ownership of the developers (i.e. who pays the salary of the key contributers of the project.) When the latter changes, having the former sorted out considerably reduces the impact. And this comes from my own personal experience dealing with Hudson/Jenkins. This is one of the reasons why the Jenkins project has a governance structure laid out.

So I’d like to encourage the Groovy project to sit down and clarify the governance. This should be a part of their “find a new home” plan. Maybe they could even just join Apache Software Foundation since the license is compatible, maybe they could come over to SPI, where Jenkins is.

At this point, I’d be very happy with a less active Groovy project without a corporate sponsorship. But I wouldn’t like to see a governance-less Groovy project. I think they can avoid that.

Update: I’ve written a follow-up post on this topic

Why I hate Zendesk

In Jenkins project, I use JIRA. And for CloudBees customers, we use Zendesk. And I positively and passionately hate Zendesk. It’s quite painful for my usage, and even more disappointing as many of the things that make me suffer can be fixed so easily. So I decided to write this to make myself feel better.

The way we use Zendesk, it’s really not that diffrent from the way Jenkins JIRA is used, which probably applies to just about any software projects. People report strange behaviours in software, stack traces, log files, etc. We look at those, sometimes ask for more information, ask them to try a new binary, get a new data, and repeat this process until it runs its course. I don’t think we are that unique.

This kind of support work is a lot like a detective work. You look at various log files and stack traces, you connect dots, and you derive a certain hypothesis. Sometimes additional facts are discovered later, or a hypothesis is proven wrong, and you start digging in another direction.

When I’m doing this, it’s very important that I “cite” my sources correctly. I’d like to be able to say “the stack trace in comment #7 indicates that X is doing Y when Z. This is consistent with ticket #150 comment #9.” But I can’t do this in Zendesk, because they do not have any sequence number nor ID for comments. So the best I can say is “comment made by John 11:59 on Apr 13th 2013 in ticket #150″, and if somebody (like me 6 months from now) wants to see the referenced comment, he has to manually find that comment. This is like writing a research paper without a bibliography section, or how a sloppy program managers refer to tickets (“what happened to that ticket — you know, the stability issue from Acme Corp a few weeks ago?”). This is particularly maddening because obviously they have an unique ID in their database, and I can even see it in the HTML source. It’s really only a minor additional work to make comments referencible. And yes, JIRA has permalinks for comments for long time.

There’s a lot of useful features to be built in this direction — for example, I’d love to see log files visible online, in such a way that each line is separately addressible as URL, and allow agents to left comments, much like how you review code. In this way, I can leave the audit trail for my detective work so that others can retrace that later.

Another thing that bugs me is the inability to edit comments that have already been posted. Zendesk supports markdown, which is good; it helps you organize information a lot more than just plain text. But some customers don’t know that, and file a ticket in plain text. Similarly, when the ticket comes from the e-mail gateway, often line wrapping and etc messes up the format.

In JIRA I often edit the ticket description to fix this up — convert a section of the text to a fixed-width font, remove line wraps, delete the pointless e-mail footers, etc. Have you ever had someone who posted 1500 lines of log records as a comment? Move that off to an attachment, and edit that out! In other situations, I want to go back to my earlier comment and strike out something, for example when my earlier hypothesis turns out to be wrong. But I can’t do that either in Zendesk. (And Zendesk doesn’t let you collapse a comment, either.)

I can go on forever, but I should be working, not whining, so I’ll finish this up with just one more problem; inability to edit a closed ticket.

Since CloudBees uses Zendesk for supporting paid customers, developers are expected to bring tickets to completion, to the “closed” state. I’m pretty sure we aren’t unique. The problem is that once a ticket is closed, one cannot make any further updates.

Say another ticket was filed later by somebody else that is seemingly related? Can’t add a comment to link to it. If a detective work led to an old “unresolved” case and you came up with a new hypothesis that might explain a behaviour? Can’t put it there. Adding a new tag? Nope.

My colleague Ben Walding made an observation that Zendesk probably has a different kind of audience in mind — ones where a relatively unsophisticated “level 1″ support people sitting in an office somewhere in Asia goes through 100s of tickets every hour as fast as possible. In other words, Comcast or AT&T kind of support. Prominence given to features like a templated response backs up this assumption. I can only assume Zendesk works well for those use cases.

But if you are considering Zendesk and your use case is more like ours, the sophisticated software support, then I hope you learn from our mistake and stay away from Zendesk.

LEGO Earth project

I play LEGO a lot with my daughter, and I’ve been obsessed with making spheres. My last sphere building attempt was building a mini LEGO Earth in 2009. While I enjoyed the project, it was also clear to me that there is a room for improvements.

The main issue is that the the “pixel resolution” of LEGO is rather low that many of the distinctice coast line shapes are just not visible. I also later discovered off-by-one bug in the program that I used to compute the color assignment to tiles, which skewed the projection around seams.

So around March last year, I decided to build a bigger version of it, at the whopping 3x scale of the earlier project. This will surely give me enough resolution!

The construction starts with building a 36x36x36 stud cube from LEGO technic beams. The studs are facing all 6 directions.

The lumps at corners are for changing the stud direction as well as for supporting the structure.

The idea is to basically build six peels and put them on the surface of this cube, completely covering the cube:

Each peel is big enough that it needs its own support structure. When attached to the cube, the center of each peel will not be connected to anything, so it needs to be fairly sturdy. It would be fun to write another program that does the structural analysis and figure out how much support would be adequate. Perhaps a TODO for the next version. The clipboard you see on the left is the instruction computed by my program

The support structure is mostly made of 2×4 bricks. Just for this purpose, I bought several 1000s of them:

On this support structure, I then built up the arch over it. One section is almost complete!:

One thing I wasn’t too careful about in the planning phase was the seasonal change of the Earth surface.

The image I based the texture mapping on didn’t have any sea ice, so I use a different source to add in sea ice. Unfortunately, those two sources did not agree on the season. If you look carefully, the sea ice is the winter level, but the land ice is the summer level:

On the positive side, the higher resolution really paid off. Here are two Earths side by side, showing the same side.

We are used to seeing the Earth as a two dimensional map projection, so our mental image of the Earth is quite skewed. Building an actual sphere was quite an enlightening experience, as I get to really look at the actual proportions of things. For example, I used to think that when I fly from the US to Europe, I’m going about one third of the Earth, but it turns out it’s more like just one fourth. Africa is really, really large, and the mouth of Amazon river is so vast that the brown area caused by the river-carried dirt stretches a hundred kilometers. For that I gave is a distinctive brown tile:

This version of Earth is about one meter across. It’s quite heavy, but I can still carry it by myself.

Once I assembled it, I still had some more fun by calculating various numbers based on this scale.

For example, the highest point of the Earth, Mount Everest in Himalayas, is only about 0.3mm tall above the sea level. That is, it’s not even one tenth of a LEGO plate high. In fact, he whole Earth crust is only about 30km deep, so that’s only a half LEGO plate high. If I wanted to correctly model the interior of the LEGO Earth, then I have to basically build a red sphere all the way, except the top-most plates. It’s almost like we are living on the egg shell, and no wonder the continents moves over time!

In the same scale, the Sun would be about 50m (200ft) in diameter at 6km / 4miles away. That’s big enough to comfortably fits the Statue of Liberty inside. Someone please build that in LEGO and we will put that and my Earth together at the accurate distance!

Finally, if you put this Earth in my home at San Jose, the furthest man-made object, Voyager 1, is past San Diego and into Mexico, flying at the speed of 1.5m/h. Picture a garden snail that crawls past the LEGO earth, slow that down 100 times, and that’s about the speed of Voyager 1. No wonder it took 35 years to get there — go Voyager!

Anyway, that’s it. Now that I’m finished with this project, what am I going to build next?