The Problem with Subtle Shades of Mu

This post was provoked by a real event,

unlike many of my posts which are born of pure imagination.

Javascript distinguishes between null and undefined. A cursory Google suggests that null is meant to be used by the programmer to represent an intentional null value and undefined is supposed to be a value that “doesn’t exist”. But this is a pretty vaguely worded distinction, and it lives entirely in the programmer’s mind and culture. The compiler has no idea what the difference between the two use cases is.

I was using a webpage a while ago and noticed that one of their buttons didn’t work. (It was a pretty important button too, rendering the page completely unusable.) On a whim, I decided to open up the Firefox dev tools console and see whether there was some kind of error. Sure enough, TypeError: paymentMethod is undefined showed up in the log, with a handy line number attached. I clicked on the line number and scanned through the code, wondering if the fix would be obvious, and… wait a minute, they’re checking whether paymentMethod is undefined just a few lines above! Control flow should never hit here unless paymentMethod has a value…

Oh wait. The check is paymentMethod !== null. And sure enough, undefined == null evaluates to true, but undefined === null evaluates to false. They checked to make sure paymentMethod wasn’t null, and then acted as if they knew paymentMethod was neither null nor undefined — relying on assumptions that were not provided by the check they made.1

if (x !== null) ... is an insidious mistake, because it looks like within its scope you should be safe. Even an experienced programmer could be forgiven for allowing their eyes to gloss over it, and proceeding to assume that in the following code block x has a value.

Some language features can demand an unreasonable amount of your attention.

A vast maze of choice

Clojure has two falsy values, false and nil. false is a pretty straightforward boolean; nil is a bizarre mashup of falsity, failure, every kind of empty sequence, and Java’s null.

It just so happens that idiomatic Clojure uses a lot of functions that return “either nil or some non-nil value” as pseudo predicates, since you can conditionally branch on any kind of value.

But nil corresponds to null on the Java side, so it generally shouldn’t be used with Clojure’s java interop. I once had to write (if x x false)2 instead of x, because we had a function that called out to Java and therefore wasn’t allowed to return nil. (I’m lucky the function’s docstring said as much, or it could have taken me weeks to figure out what was going wrong.)

And then there’s the fact that a lot of Clojure data is represented by key-value maps. What’s the difference between {:some-property false}, {:some-property nil}, and {}?3 Are they all allowed? Do you prefer one form over the others? You decide!

Or well, you try to decide. But it’s not really your choice: it’s your codebase’s choice. Every time someone checks to see whether a given map has :some-property or not, they’re making an implicit choice as to what those three things mean. And if sometimes people are sloppy, because in the heat of the moment one interpretation was obvious and the others never even came to mind… well, you just have to hope that all of these accidental, unconscious choices happened to match up.

(Even though you know in your heart that they don’t. They never do, for the universe is cruel.)

Whatever shall we do?

If you’ve been reading my blog for very long, you might expect me to now embark on a rant about how Haskell does it better. This is, in fact, what I’m going to do.4

For very simple sources of failure, there’s the Maybe type, which I have probably written a lot about (and if it doesn’t have its own dedicated blog post it will probably get one eventually). Maybe is useful when there’s only one kind of failure possible (or there might as well be only one for all you care), and when you want to respond to that failure just by abandoning the control flow that led to the error. When you’re in this situation, you can use Maybe to introduce a single failure value, guaranteed to be distinct from any of the “success” values, which basically has the behavior of causing control flow to be abandoned whenever something tries to operate on it.

What about when you want to have multiple failure values, with subtle distinctions between them?

Well, first of all, think twice. Dealing with subtly different shades of mu can be a huge headache.

But if you really need to, Haskell has an idiomatic way of easing the headache. First, define an explicit data type enumerating the ways something might have failed: data Error = KeyboardNotFound | SegmentationFault | PEBKAC. Then you use the Either type, which is like Maybe except that instead of one failure value there are many. (In particular, Either errorType introduces as many failure values as there are values of errorType). But while there are many different failure values, there’s a bastion of order and sanity for you to cling to: all of the different ways of failure are guaranteed to have the same effect on control flow (namely, the abortion of control flow). And it is guaranteed to be distinct from the effect of successful values on control flow, which are in turn all the same as each other.

And most importantly, there is a single way to check for this composite notion of failure, and this check is easy to remember and doesn’t hide in a room full of distracting wrong alternatives.

  1. this is one of the more important, nonobvious things a static type system can give you. A really good type checker can, to some degree, infer what assumptions a code block relies on and what properties a dynamic check verifies. If these things don’t match up, as in this javascript case, you can get an error at compile time. 
  2. I now realize that this could be replaced by the equally horrible but more compact (or x false)
  3. It gets even worse if you think about lists instead of booleans. false vs nil vs “not present” works out the way you’d expect barring occasional subtle interactions, but [] vs () vs {} vs #{} vs “not present” requires a lot more care — especially since almost all functions that act on collections return seqs5 (like ()) regardless of what the input was, and because nil can be treated to some extent like an empty collection of any built-in collection type. 
  4. note, however, that this section glosses over a few things. It is in praise of Haskell’s ideal philosophy, which I do believe is closer to perfection than any language before it, but which Haskell does not unfailingly embody. I’m happy to talk about the various unfortunate departures Haskell makes, but in this section of this blog post a lengthy discussion of such would merely distract from the point. 
  5. except, ironically, for seq, which coerces the input collection to a seq unless the input was empty in which case it returns nil

Failing, Simplicity, and Failing Again

As you might have noticed, Project Serpent has stalled. Between work, the errands of daily life, and other recreational pursuits, I’ve simply spent enough time choosing not to work on Serpent that it has fallen away from the top of my mind.

And that’s not the only thing that has fallen away. Without Serpent updates to post, I’ve had only a few scattered programming-related thoughts, each of which would have taken a lot of effort to turn into a blog post. Ideally I’d put in that effort, but lately I haven’t. And due to the title of this blog, I’ve been reluctant to write posts about the various non-programming things I’ve been thinking about. (I might change the title of the blog in light of this… but it can stay for now.)

So I haven’t been working on my projects, and I haven’t been writing blog posts. I’m not happy with this, and I’m trying to change my behavior.

The first, easiest, and most obvious thing is to write a blog post. That’s simple enough that with a little discipline I can just decide to do it, and it’s done. So here we are. Let’s hope I do this again next Sunday.

Getting back to work is harder. It’ll need discipline too, but pure discipline isn’t enough. I need a plan that compliments my willpower.

The best idea I have now for rekindling my motivation is to go back to something simple. In fact, since my goal is still to return to Project Serpent, I decided I’d go with something even simpler and quicker to build than Pong. I set myself the constraint of making a one-button game that I could feasibly finish in a single day.

Since thinking is a lot easier than programming, I even put some thought into what that game would be like on the train to work. The standard concept for a “one-button game that still manages to be enjoyable” is an obstacle course to jump around, like Canabalt. But I don’t find this model compelling, and I wanted to do something that I hadn’t seen before.

My first few ideas were impractical, either deriving their interestingness from monumental complexity or turning out not to make sense for a one-button game. Eventually I hit on the idea of a rhythm game as the primary mechanic; and that doesn’t fulfill the “something I haven’t seen before” criterion but at least it’s an example of something that isn’t a running game.

And in fact, a slight mutation of this gave me my major breakthrough. It occurred to me that I could throw multiple objects at the player at once by giving each one a kind of “resonant frequency”, and then the player can decide which object to react to at any given time by tapping in resonance with them. Then, the game is about (1) deciding which of a bunch of random enemies to deal with, (2) eyeballing the target frequency of those enemies, and (3) tapping out that frequency to get rid of them.

This felt perfect — it was still based on simple concepts in the spirit of a one-button game, but the idea seemed to open up deep and engaging gameplay, and most importantly I was finding myself motivated to make the game. I wanted to see how it would play. I wanted to play it myself.

So I mentally prepared myself, for the rest of last week, to spend a day on this project. I was excited to finish and see how the idea worked. I thought about showing it to my coworkers at Prismatic (and I’m still planning to be able to do that soon).

I even woke up early on Saturday, and booted up my laptop and text editor and started typing.

Except… I didn’t feel it pulling together. Everything felt so much more effortful than I was used to, code failed to typecheck, and I started to question my model and how it would be used to implement the game I was envisioning. And after a few hours I realized I’d made no progress and decided I couldn’t do it.

That was not this past weekend, but the 10th and 11th.

I have a few ideas for getting started again. My ideas don’t often seem to work… but I remember Pong, and I know I can do this somehow. What can I do but keep trying?

What I Gained from Hacker School

I spent the last two and a half months at Hacker School, which has been described as a “writer’s retreat for programmers”. You fly out to New York to their space, where you join a bunch of other programmers, providing social pressure to do some programming every day and a community of people with whom you can talk about programming. For a solitary person such as myself, both of these are extremely helpful and novel.

Hacker School is also completely free to attend and even has cost of living grants for a steadily growing set of demographics. Their business model is like that of a recruiting agency: they help out any Hacker Schooler who is looking for a job and charge companies for hiring these people. For me, this sounded almost too good to be true: having never had a job, I was desperate for help breaking into the technology industry through the wall of college degree and years-of-experience requirements. But there was no way my parents could afford to fly me all the way out to New York and put me up in an apartment for two and a half months.

So I went to Hacker School for two reasons: to experience and environment where programming is a normal part of life, and to get a programming job. What did I actually get from Hacker School?

First and most exciting: I did get a job. The wonderful Hacker School facilitators, especially Sonali Sridhar, guided me through the process of writing a resume that is impressive despite its lack of experience, and of reaching out to Hacker School’s various partner companies. I got more interviews in a couple weeks than in the six months I had spent optimistically looking for a programming job on my own. And while most of these leads eventually turned me down, one company seemed consistently impressed by me and kept inviting me back… and on December 1st I start with Prismatic. When I got accepted to Hacker School I hardly dared hope for an outcome this good, and it definitely wouldn’t have happened if not for Hacker School.

There are also some things I gained that I hadn’t really anticipated.

For example: I have been wanting to start a blog for at least a year, probably longer, but never been able to commit to writing regular posts. There might even be a handful of dead WordPress or Blogspot blogs of mine scattered around the web — I don’t remember any of them, and certainly there are none with more than one post.

But at Hacker School I started to realize that people like hearing what I have to say. I even gave a little 20 minute talk and the handful of people who wanted to hear it seemed to enjoy it. And at the same time, I got together with some other Hacker Schoolers who were also trying to commit to writing one blog post a week. At first I thought this attempt would be like all the others, that I’d make a handful of posts and then forget again. But it’s been three months and I’m still posting… and people are even reading.

And I didn’t produce anything at Hacker School that I’d call finished, but that’s normal: I hadn’t really finished any projects before Hacker School either. What isn’t normal is that after Hacker School, I noticed this deficiency and corrected it. I can’t help but think that Hacker School’s atmosphere of programming every day had something to do with this sudden burst of productivity.

Finally, there’s something I feel like I could have gained from Hacker School that I didn’t. I met a lot of people there that I had fun interacting with, people who I would have been happy to get to know better and who probably would have been happy to know me… but I didn’t come away with any new close relationships, mainly because I have difficulty initiating and sustaining interactions. But even here it’s not too late: the Hacker School community is broad, and there are resources in place for helping alums connect with other alums and with current hacker schoolers. I hope to make a bit more effort along these lines as soon as I can work myself up to it!

Finally, did I mention that Hacker School is always looking for new applicants? If anything I’ve talked about sounds interesting, check out their site — it could cost you nothing at all!