Science for busy people

5 thoughts
last posted Dec. 13, 2015, 2:20 p.m.

1 earlier thought

1

Why we can't do science: a parable

Say we're going to write a new computer program. Programmer Zero says we should write it in Rust, Programmer One says we should write it in Haskell. Programmer Two says we should be scientific about this.

Zero: Let's write our new web app in Rust!

One: No, let's write it in Haskell, it's much more mature.

Zero: Rust has much better tooling and documentation, and is has a gentler learning curve.

One: Haskell's awesome type system will prevent bugs before they happen. Besides, we don't need a systems programming language for a task like this.

Zero: Yeah, but nobody uses Haskell. Rust, on the other hand, was recently used for $AWESOME_THING and they said their experience of it was great and they just IPOd for five billion dollars.

Two: We shouldn't make such important decisions lightly. Let us be scientific about this.

Zero: OK, let's do an experiment. I'll spend a month writing it in Rust, while at the same time One can spend a month writing it in Haskell. At the end of the month, we'll see which is better.

Two: But how will we decide which is better?

One: Well, I guess which one is most feature complete, and performs the best.

Two: Performs the best how?

One: Since it's a user-facing app, I guess latency is the best metric?

Two: Maximum latency? Latency for all requests? Latency under what load? With how many servers? Would the Go one be better if it had lower maximum latency but required twice as many servers?

Zero: Yeah, also, we haven't mentioned bugs. Or maintainability. I don't even know how to measure that.

One: And I've got to do twelve interviews over the next month, and I'm on holiday next week. Maybe we'd get to the end of it and decide that Rust was better but actually it was just because I just couldn't do as much work.

Two: So we can't make a decent decision about which is better. Therefore, we should do it in Python, which is what we've used for everything else.

Zero: That doesn't make any sense.

(enter Manager)

Manager: I just went to a conference and everyone was talking about Go. Let's write the new thing in Go—I want to see it live in two weeks. Also, can we make it a microservice?

3 later thoughts