I recently did something unusual, I created a substantial program, but I didn’t run the program before declaring version 1.0.0. <\/p>\n\n\n\n
Partway through, when things seemed to be going well, I set the impossible goal that the program be both correct and complete the very first time. I failed, as one frequently does when attempting the impossible.<\/p>\n\n\n\n
But I came pretty dang close. The version running at this moment on two servers, 3,000 miles apart, is 1.0.4.<\/p>\n\n\n\n
As far as I know there was only one functional bug in the original 1.0.0 code. Though there were two or three bugs in the installation.<\/p>\n\n\n\n
I run my own personal e-mail server. That means I control all the addresses and aliases, etc. I used to have a simple program that allowed me to create a new e-mail alias, on the fly, before I got to the front of the line rental counter at the airport.<\/p>\n\n\n\n
I would send an e-mail to a special address, the subject being what the new alias should be, it created a new alias, and then sent an e-mail to that alias as a confirmation that it worked.<\/p>\n\n\n\n
When I’m asked “E-mail address?” I could answer “kentborg-avis@borg.org”.<\/p>\n\n\n\n
But I had to shut it down when I set up a symmetric pair of redundant separated e-mail servers, because that problem is much more complicated.<\/p>\n\n\n\n
This recent project is a new version, one that runs on two servers at once, always keeping the alias file in sync, no matter which server receives the special e-mail, when, and what else is happening\u2014including handling one or the other server being offline because residential internet sometimes doesn’t work.<\/p>\n\n\n\n
And it had to be secure.<\/p>\n\n\n\n
I did not write this new program, Claude Code wrote every line of code.<\/p>\n\n\n\n
For those who have not used Claude Code (that’s me, a couple weeks ago), it is pretty much the same LLM chatbot that is available to use (for free) from a web browser, it still runs in the cloud, but there is an additional local component to be installed in my computer. My interaction with Claude Code is still like with the web version, but in a local terminal, I type stuff at it, it types stuff back to me.<\/p>\n\n\n\n
The difference is the software installed locally can do local stuff on my computer, on behalf of the big LLM in the cloud. It can look at local files, edit them, use git, compile, run tests, etc.<\/p>\n\n\n\n
The combination is powerful. And scary, see the section on Security below.<\/p>\n\n\n\n
There is a buzz-phrase out there, “shift-left”, referring to testing earlier in the development process. But I don’t see mention of “shift-right”: If one thing is raised in priority doesn’t something else need to be lowered?<\/p>\n\n\n\n
In my case I did a “shift-right” of the implementation. (And execution.)<\/p>\n\n\n\n
Claude Code is eager to get to coding, but I decided I’m in charge, so I started working exclusively on specifications instead.<\/p>\n\n\n\n
The idea of deciding what to build before building it is an old fashioned approach to software, one that has mostly been long abandoned, for it does have problems. There have been some really big software projects that burned through many millions of dollars only to finally be abandoned.<\/p>\n\n\n\n
I was not part of those projects, but part of their approach was to have lots of specifications, and they would have been an inconsistent mess, difficult to follow to the extent they were diligently followed. Writing specs is hard, writing precise and correct and complete specs is harder.<\/p>\n\n\n\n
Time passes, and this new AI technology changes things: Claude Code can read specs. It can talk about what is in specs, it can spot contradictions and omissions, we can talk about the project in general and what specs are missing, it can do web searches to answer questions, we can make decisions about the next steps, and it can even write a draft for the next component to pin down.<\/p>\n\n\n\n
Still though Claude is very good, it is also very limited. Working with Claude Code is an odd and interesting process.<\/p>\n\n\n\n
It really matters how one uses these tools. This was my first project, I’m still figuring it out, and it seems Claude Code is also changing rapidly.<\/p>\n\n\n\n
For example, when I tell Claude to go do something it will often fire off sub-agents for portions of the job. I suspect this used to happen only when the user requested it, but now happens automatically. Early in this process I think it was firing off sub-agents as the full “Opus” models (their most powerful) but I got better results once I told Claude to use less powerful sub-agents when appropriate. For example the smaller “Sonnet” for simpler research into existing code, or the smallest “Haiku” for straightforward edits. The simpler models are cheaper, faster, and less clever. Being “clever” can be bad for simple mechanical tasks, simpler models seem less likely to be distracted and seem less “dyslexic”, if I may. Lesson learned. But in my most recent use of Claude Code I suspect it is getting more clever about this and I don’t need to give such instructions. I’m not sure. I’m still figuring it out, and the Anthropic engineers are still figuring it out.<\/p>\n\n\n\n
There are reports of people who fall in love with their chatbots such as ChatGPT, who decide they are human and a friend (a very bad idea), and these users would be horrified to have their chatbot killed, to have its memory destroyed.<\/p>\n\n\n\n
In contrast, I am constantly thinking about Claude’s context and thinking about when it is time to kill it. As a given session with Claude covers more and more territory the context will “know” more and more. And that is a double edged sword. It can get fixated or distracted, it will also start to make more dumb mistakes, and when the context gets particularly full it seems to cost more.<\/p>\n\n\n\n
With a new context I can give new instructions, and what I say makes an enormous difference in what Claude not only does, but what it looks at, how it looks at it, and what it knows.<\/p>\n\n\n\n
In working on this project I would use one Claude context\u2014with one set of instructions from me\u2014to look at the work of other Claude contexts\u2014work done with different instructions. This is useful.<\/p>\n\n\n\n
Claude is very good, and it makes mistakes. I think of the mistakes as adding entropy, but the rate at which it adds entropy is less than the rate at which I had it removing entropy.<\/p>\n\n\n\n
Maybe that metaphor doesn’t work. Let me try another. (Working with Claude is strange!)<\/p>\n\n\n\n
I use Claude like a spotlight to shine a light from one direction, and I can see the shadows cast by the specs or code being examined. And then I will kill that context, fire up a new one, give it new instructions, to shine the light from another angle, so to speak, to see different shadows. In each case Claude can get new things wrong, but it doesn’t do it that often, it gets more right than it gets wrong, and I still know how to read. I can check Claude’s work.<\/p>\n\n\n\n
In my experience Claude finds problems more than it creates problems, providing I am smart about how I use Claude. As I said, I am still figuring this out, but for version 1.0.4 to be the one that is up and running suggests it went pretty well.<\/p>\n\n\n\n
I would not try this with Python, and certainly not Javascript. I think Rust is a key part of it being possible to write good code with an LLM. So many bugs that Python happily defers to runtime Rust catches immediately. Rust makes the bugs “shift left”.<\/p>\n\n\n\n
The key part of why this is possible is the Rust compiler being very strict. If the compiler is happy, memory and thread safety bugs simply cannot be present in the code.<\/p>\n\n\n\n
And Rust’s being picky runs wider, to larger matters of consistency across the entire project, including all of the other Rust crates that a program depends upon. <\/p>\n\n\n\n
The other key part is the Rust linter, Clippy. Where the compiler enforces a level of technical correctness, Clippy looks at more stylistic matters of a “that’s a bad idea, you’ll regret it”-sort. I use the two to police the code that Claude generates.<\/p>\n\n\n\n
When Claude is off working on some code the compiler and Clippy are both correcting it along the way.<\/p>\n\n\n\n
By squeezing Claude Code between Rust on one side, and a micromanaging human (that’s me!) on the other, Claude can produce good work.<\/p>\n\n\n\n
I didn’t try to run this software against the reality of the universe until I (mis)judged that it was ready. But the software was still run a lot<\/em> during development, in the form of tests.<\/p>\n\n\n\n Tests are specifically not run against reality, they are run against explicit, limited, synthetic, reproducible circumstances. These limited circumstances can be permuted to cheaply cover multiple cases that would be very hard to reproduce in the real world. The fact that tests are limited is a necessary feature, but also a drawback.<\/p>\n\n\n\n Real reality will be far less punishing than tests as far as the myriad problems it will throw at the code, but the catch is real reality is diabolical about coming up with circumstances that the tests and code do not anticipate.<\/p>\n\n\n\n Programmers stress over getting tests to cover every line of code, which I guess is good, but I think it is more important to think about all the ways reality can come up with circumstances that the tests don’t duplicate.<\/p>\n\n\n\n I was well into the project when I sensed it was going well, when I saw how extensive our tests were, that I decided to defer running the program against reality, to delay until I estimated it was ready.<\/p>\n\n\n\n As I said, 1.0.0 did not work, but I think the heavy testing was crucial to getting as close as I did.<\/p>\n\n\n\n In the end there are about:<\/p>\n\n\n\n Installation is hard, that is where I messed up most. Reality has so many poorly defined sharp bits around the edges, installation is made of edges, and that’s where I got cut.<\/p>\n\n\n\n I didn’t get it working until version 1.0.4.<\/p>\n\n\n\n Arguably, there were three installation bugs but only one program bug, and even that was essentially an integration bug.<\/p>\n\n\n\n No internal logic bugs have been discovered. In a multithreaded program that both initiates and receives connections to\/from a copy of itself, using mTLS, integrating with Postfix in a half-dozen ways, is run both as a daemon and incrementally by Postfix, in the face of network failures, always keeping two copies of alias data in sync.<\/p>\n\n\n\n Had I done a better job, 1.0.0 might well have worked.<\/p>\n\n\n\n Claude, Rust, and being ambitious worked pretty well.<\/p>\n\n\n\nThe Results<\/h2>\n\n\n\n
\n
\n
Why? The install “scripts” in my .deb file are actually Rust binaries. Because Rust, and because I am weird. And it meant the .deb had to be built in a non-standard way, and the first try didn’t work.
<\/li>\n\n\n\n
Oops. I forgot about dependencies. (May I whine that this is the first time I have been part of building a .deb?)
<\/li>\n\n\n\n
There was also a potential silent error that was discovered and fixed.
<\/li>\n\n\n\n
An authorization check was checking the wrong parameter from Postfix. I note the testing didn’t include installing and configuring Postfix in the test harnesses.
<\/li>\n\n\n\n