It's already been a few weeks since the last update! I'm back with a new feature and some guidelines I'm trying to follow as I build.
Checks
Checks are Ubik's take on "Continuous Integration"[1]: your tests run locally and their status checks are stored in your repo. No need for a remote server to tell you whether or not you can ship your software. Here's an early look at how it works.
In the demo video, I'm using a keyboard shortcut to run the tests for my project at specific commits. This works by creating a git worktree for a given commit and running the tests on that worktree. The first one fails because I broke the tests in that commit, so I'd ostensibly want to block that commit from going to production. You'll see that the previous commits pass, so those should be good to merge. Blocking commits with failing tests from merging isn't implemented yet, but it's something I'll get to soon.
Why?
A conventional setup for software teams is to write automated tests for their code, push the tests and production code to a feature branch, wait for a code review and for the tests to pass in CI (someone else's computer), then merge your branch into the mainline. This has been the status quo for every team I've been a part of. For the past year, I've been wondering why those tests need to run on someone else's computer. Why don't we ship code that is only tested locally?
I think there are two reasons:
In the demo video, I'm using a keyboard shortcut to run the tests for my project at specific commits. This works by creating a git worktree for a given commit and running the tests on that worktree. The first one fails because I broke the tests in that commit, so I'd ostensibly want to block that commit from going to production. You'll see that the previous commits pass, so those should be good to merge. Blocking commits with failing tests from merging isn't implemented yet, but it's something I'll get to soon.
Why?
A conventional setup for software teams is to write automated tests for their code, push the tests and production code to a feature branch, wait for a code review and for the tests to pass in CI (someone else's computer), then merge your branch into the mainline. This has been the status quo for every team I've been a part of. For the past year, I've been wondering why those tests need to run on someone else's computer. Why don't we ship code that is only tested locally?
I think there are two reasons:
- Your computer is slow. You can rent bigger computers in the cloud to run your tests much faster than your machine can. That way, you can can push your code to be tested on the remote server and move on to the next task.
- Development environments are brittle and vary slightly from machine to machine. This leads to tests and behavior that "works on my machine" but not on yours. It's a massive waste of time and energy. If we can all agree that our software works when it runs correctly on a remote server, our lives become much simpler.
Let's reconsider each of those:
- The gap between dev machines and servers is closing enough for us to at least revisit the trade-offs. If you've already spent $1,500 - $3,000 on a computer and it can run your tests in less time than it takes to make a coffee, is it worth spending hundreds or thousands per month to run tests on a remote server? It might be, but it's not as cut and dry as it once was. Plus, I bet you're going to go slower by having your developers switch contexts to their next task while their tests run, especially if they don't pass.
- Fixing broken software due to subtle dependency issues is frustrasting, but I think it demonstrates a team's lack of ability to automate rather than a fundmental limitation of one's own computer compared to a server. What's so special about the server? I think you can mitigate the risks by taking on fewer dependencies, pinning their versions, then looking into running your dev environment in something like Docker or Nix. When your software is reproducible across different environments, your machine is as good as any. Obviously, this could be different for your project. You may need to use a server to emulate some hardware or system that aren't possible to emulate locally.
I've been thinking about this for a while, but I had a feeling I was crazy and that there must be better reasons for the status quo. Then DHH wrote about this very topic back in April. Aaron Jensen wrote a related piece last year. Their articles aren't evidence for the approach being correct, but at least I'm not the only one thinking about it.
Building for me
I've fallen into this trap a thousand times: I'm writing code to solve an immediate problem, but I can't help thinking about how to generalize it to fit other cases. This also happens when I'm working on projects for myself. It's tempting to think about how what you're building could solve all sorts of problems. Think of the impact you could have if you just abstract it a bit more!
I'm trying as best as I can to avoid doing this while building Ubik. For now, I'm building it for me. I may never take it any further than that. If I do, I'm not going to try to solve the problems that large teams face while building software. I care about enabling individuals and small, high-trust teams to do their best work. Building for feature parity with Github, Gitlab, and the rest is not interesting to me.
In the next devlog, I should have more technical things to talk about.
[1] Martin Fowler has a good explanation for why the conventional use of the phrase "continuous integration" has lost its original meaning. I'm going to dig further into his writing on the topic. I have a feeling it's going to inform future decisions in Ubik.