JJ LSP Follow Up

In Majjit LSP, I described an idea of implementing Magit style UX for jj once and for all, leveraging LSP protocol. I’ve learned today that the upcoming 3.18 version of LSP has a feature to make this massively less hacky: Text Document Content Request

LSP can now provide virtual documents, which aren’t actually materialized on disk. So this:

can now be such a virtual document, where highlighting is provided by semantic tokens, things like “check out this commit” are code actions, and “goto definition” jumps from the diff in the virtual file to a real file in the working tree.

Exciting!

Against Query Based Compilers

Query based compilers are all the rage these days, so it feels only appropriate to chart some treacherous shoals in those waters.

A query-based compiler is a straightforward application of the idea of incremental computations to, you guessed it, compiling. A compiler is just a simple text transformation program, implemented as a lot of functions. You could visualize a run of a compiler on a particular input source code as a graph of function calls:

Here, schematically, squares are inputs like file text or compiler’s command line arguments, g is an intermediate function (e.g, type checking), which is called twice, with different arguments, and f and h are top-level functions (compile executable, or compute completions for LSP).

Looking at this picture, it’s obvious how to make our compiler “incremental” — if an input changes, it’s enough to re-compute only the results on path from the changed input to the root “query”:

A little more thinking, and you can derive “early cutoff” optimization:

If an input to the function changes, but its result doesn’t (e.g, function type is not affected by whitespace change), you can stop change propagation early.

And that’s … basically it. The beauty of the scheme is its silvery-bullety hue — it can be applied without thinking to any computation, and, with a touch of meta programming, you won’t even have to change code of the compiler significantly.

Build Systems à la Carte is the canonical paper to read here. In a build system, a query is an opaque process whose inputs and outputs are file. In a query-based compiler, queries are just functions.

The reason why we want this in the first place is incremental compilation — in IDE context specifically, the compiler needs to react to a stream of tiny edits, and its time budget is about 100ms. Big-O thinking is useful here: the time to react to the change should be proportional to the size of the change, and not the overall size of the codebase. O(1) change leads to O(1) update of the O(N) codebase.

Similar big-O thinking also demonstrates the principal limitation of the scheme — the update work can’t be smaller than the change in the result.

An example. Suppose our “compiler” makes a phrase upper-case:

compile("hello world") == "HELLO WORLD"

This is easy to incrementalize, as changing a few letters in the input changes only a few letters in the output:

compile("hallo world") == "HALLO WORLD"

But suppose now our “compiler” is a hashing or encryption function:

compile("hello world") == "a948904f2f0"
compile("hallo world") == "a7336983eca"

This is provably impossible to make usefully incremental. The encryption can be implemented as a graph of function calls, and you can apply the general incremental recipe to it. It just won’t be very fast.

The reason for that is the avalanche property — for good encryption, a change in any bit of input should flip roughly half of the bits of the output. So just the work of changing the output (completely ignoring the work to compute what needs to be changed) is O(N), not O(1).

A particularly nasty effect here is that even if you have only potential avalanche, where a certain kind of change could affect large fraction of the output, even if it usually doesn’t, your incremental engine likely will spend some CPU time or memory to confirm the absence of dependency.

In my

Three Architectures For Responsive IDE, query-based compilation is presented as a third, fall-back option. I still think that that’s basically true: as a language designer, I think it’s worth listening to your inner Grug and push the need for queries as far down the compilation pipeline as possible, sticking to more direct approaches. Not doing queries is simpler, faster, and simpler to make faster (profiling a query-based compiler is a special genre of hurdle racing).

Zig and Rust provide for a nice comparison. In Zig, every file can be parsed completely in isolation, so compilation starts by parsing all files independently and in parallel. Because in Zig every name needs to be explicitly declared (there’s no use *), name resolution also can run on a per-file basis, without queries. Zig goes even further, and directly converts untyped AST into IR, emitting a whole bunch of errors in the process (e.g, “var doesn’t need to be mutable”). See Zig AstGen: AST => ZIR for details. By the time compiler gets to tracked queries, the data it has to work with is already pretty far from the raw source code, but only because Zig language is carefully designed to allow this.

In contrast, you can’t really parse a file in Rust. Rust macros generate new source code, so parsing can’t be finished until all the macros are expanded. Expanding macros requires name resolution, which, in Rust, is a crate-wide, rather than a file-wide operation. Its a fundamental property of the language that typing something in a.rs can change parsing results for b.rs, and that forces fine-grained dependency tracking and invalidation to the very beginning of the front-end.

Similarly, the nature of the trait system is such that impl blocks relevant to a particular method call can be found almost anywhere. For every trait method call, you get a dependency on the impl block that supplies the implementation, but you also get a dependency on non-existence of conflicting impls in every other file!

Again, refer to the Three Architectures for positive ideas, but the general trick is to leverage language semantics to manually cut the compilation tasks into somewhat coarse-grained chunks which are independent by definition (of the source language). Grug builds an incremental map-reduce compiler for his language:

• Recursive directory walk finds all files to be compiled.

• In parallel, independently, each file is parsed, name-resolved, and lowered. As much as possible, language features (and errors) are syntax driven and not type driven, and can be processed at this stage.

• In parallel, a “summary” is extracted from each file, which is essentially just a list of types and signatures, with function bodies empty.

• Sequentially, a “signature evaluation” phase is run on this set of summaries, which turns type references in signatures into actual types, dealing with mutual dependencies between files. This phase is re-run whenever a summary of a file changes. Conversely, changes to the body of any function do not invalidate resolved signatures.

• In parallel, every function’s body is type-checked, and lowered to type-and-layout resolved IR, applying function-local optimizations.

• Sequentially, a thin-lto style set of analyses are run on compiled functions, making inlining decisions and computing call-graph dependent attributes like function purity.

• In parallel, each function is codegened to machine code with unresolved references to other functions (relocations).

• Sequentially, functions are concatenated into an executable file, receiving an address.

• In parallel, all relocations are resolved to now known addresses.

The above scheme works only if the language has a property that changing the body of function foo (not touching its signature) can’t introduce type errors into an unrelated function bar.

Another trick that becomes less available if you blindly apply queries are in-place updates. Consider a language with package declarations and fully qualified names, like Kotlin:

package org.example

fun printMessage() { /*...*/ }
class Message { /*...*/ }

A compiler for this language probably wants to maintain a map of all public declarations, where the keys are fully qualified names, and values are declarations themselves. If you approach the problem of computing this map with query eyes, you might have a base per-file query that returns a map of file’s declarations, and then a recursive per-directory query. And you’ll probably have some kind of structural sharing of the maps, such that changing a single file updates only the “spine”, without actually copying most of the other entries.

But there’s a more direct way to make this sort of structure responsive to changes. You need only two “queries” — per file, and global. When a file changes, you look at the previous version of the map for this file, compute a diff of added or removed declarations, and then apply this diff to the global map.

Zig is planning to use a similar approach to incrementalize linking — rather than producing a new binary gluing mostly unchanged chunks of machine code, the idea is to in-place patch the previous binary.

If you like this article, you might be interested in some other adjacent stuff I’ve written over the years, roughly in the order of importance:

• Three Architectures for a Responsive IDE
• Durable Incrementality
• Zig Language Server And Cancellation
• Resilient LL Parsing Tutorial
• Find Usages
• The Heart of a Language Server
• How to Make a 💡?
• LSP could have been better
• On Modularity of Lexical Analysis
• Why an IDE?

Wrapping Code Comments

I was today years old when I realized that:

• Code and code comments ideally should be wrapped to a different column.
• For comments, the width should be relative to the start of the comment.

It’s a good idea to limit line length to about 100 columns. This is a physical limit, the width at which you can still comfortably fit two editors side by side (see Size Matters). Note an apparent contradiction: the optimal width for readable prose is usually taken to be narrower, 60–70 columns. The contradiction is resolved by noticing that, for code, indentation eats into usable space. Typically, code is much less typographically dense than prose.

Still, I find comment blocks easier to read when they are wrapped narrower than the surrounding code. I want lines to be wrapped at 100, and content of comments to be wrapped at 70 (unless that pushes overall line to be longer than 100). That is, I want layout like this (using 20/30 rulers instead of 70/100, for illustrative purposes):

// Top level comments
// can be this wide.
const S = struct {
    // Nested comments are
    // also this wide, but
    // are shifted right.
    fn f() void {
        switch (value) {
            0 => {
                // But there is
                // a hard limit.
            }
        }
    }
}

This feels obvious in retrospect, but notably isn’t be well-supported by the tools? The VS Code extension I use allows configuring dedicated fill column for comments, but doesn’t make it relative, so indented comment blocks are always narrower than top-level ones. Emacs M-q also doesn’t do relative wrapping out of the box!

Aside on hard-wrapping: should we bother with wrapping comments at all? Can’t we rely on our editor to implement soft-wrapping? The problem with soft-wrapping is that you can’t soft-wrap text correctly without understanding its meaning. Consider a markdown list:

A list:
  * item one,
  * item two.

If the first item is long enough to necessitate wrapping, the wrapped line should also be indented, which requires parsing the text as markdown first:

A list:
  * item one which is long enough
    necessitate wrapping,
  * item two.

Diagnostics Factory

In Error Codes For Control Flow, I explained that Zig’s strongly-typed error codes solve the “handling” half of error management, leaving “reporting” to the users. Today, I want to describe my personal default approach to the reporting problem, that is, showing the user a useful error message.

The approach is best described in the negative: avoid thinking about error payloads, and what the type of error should be. Instead, provide a set of functions for constructing errors.

To give a concrete example, in TigerBeetle’s tidy.zig (a project-specific linting script, another useful meta-pattern), we define errors as follows:

const Errors = struct {
    pub fn add_long_line(
        errors: *Errors,
        file: SourceFile,
        line_index: usize,
    ) void { ... }

    pub fn add_banned(
        errors: *Errors,
        file: SourceFile,
        offset: usize,
        banned_item: []const u8,
        replacement: []const u8,
    ) void { ... }

    pub fn add_dead_declaration(...) void { ... }

    ...
};

and the call-site looks like this:

fn tidy_file(file: SourceFile, errors: *Errors) void {
    // ...
    var line_index: usize = 0;
    while (lines.next()) |line| : (line_index += 1) {
        const line_length = line_length(line);
        if (line_length > 100 and !contains_url(line)) {
            errors.add_long_line(file, line_index);
        }
    }
}

In this case, I collect multiple errors so I don’t return right away. Fail fast would look like this:

errors.add_long_line(file, line_index);
return error.Tidy;

Note that the error code is intentionally independent of the specific error produced.

Some interesting properties of the solution:

• The error representation is a set of constructor functions, the calling code doesn’t care what actually happens inside. This is why the error factory is my default solution — I don’t have to figure out up-front what I’ll do with the errors, and I can change my mind later.
• There’s a natural place to convert information from the form available at the place where we emit the error to a form useful for the user. In add_banned above, the caller passes in a absolute offset in a file, and it is resolved to line number and column inside (tip: use line_index for 0-based internal indexes, and line_number for user-visible 1-based ones). Contrast this with a traditional error as sum-type approach, where there’s a sharp syntactic discontinuity between constructing a variant directly and calling a helper function.
• This syntactic uniformity in turn allows easily grepping for all error locations: rg 'errors.add_'.
• Similarly, there’s one central place that enumerates all possible errors (which is either a benefit or a drawback).

A less trivial property is that this structure enables polymorphism. In fact, in the tidy.zig code, there are two different representations of errors. When running the script, errors are directly emitted to stderr. But when testing it, errors are collected into an in-memory buffer:

pub fn add_banned(
    errors: *Errors,
    file: SourceFile,
    offset: usize,
    banned_item: []const u8,
    replacement: []const u8,
) void {
    errors.emit(
        "{s}:{d}: error: {s} is banned, use {s}\n",
        .{
            file.path, file.line_number(offset),
            banned_item, replacement,
        },
    );
}

fn emit(
    errors: *Errors,
    comptime fmt: []const u8,
    args: anytype,
) void {
    comptime assert(fmt[fmt.len - 1] == '\n');
    errors.count += 1;
    if (errors.captured) |*captured| {
        captured.writer(errors.gpa).print(fmt, args)
            catch @panic("OOM");
    } else {
        std.debug.print(fmt, args);
    }
}

There isn’t a giant union(enum) of all errors, because it’s not needed for the present use-case.

This pattern can be further extended to a full-fledged diagnostics framework with error builders, spans, ANSI colors and such, but that is tangential to the main idea here: even when “programming in the small”, it might be a good idea to avoid constructing enums directly, and mandate an intermediate function call.

Two more meta observations here:

First, the entire pattern is of course the expression of duality between a sum of two types and a product of two functions (the visitor pattern)

fn foo() -> Result<T, E>;

fn bar(ok: impl FnOnce(T), err: impl FnOnce(E));

enum Result<T, E> {
    Ok(T),
    Err(E),
}

trait Result<T, E> {
    fn ok(self, T);
    fn err(self, E);
}

Second, every abstraction is a thin film separating two large bodies of code. Any interface has two sides, the familiar one presented to the user, and the other, hidden one, presented to the implementor. Often, default language machinery pushes you towards using the same construct for both but that can be suboptimal. It’s natural for the user and the provider of the abstraction to disagree on the optimal interface, and to evolve independently. Using a single big enum for errors couples error emitting and error reporting code, as they have to meet in the middle. In contrast, the factory solution is optimal for producer (they literally just pass whatever they already have on hand, without any extra massaging of data), and is flexible for consumer(s).

Justifying text-wrap: pretty

Something truly monumental happened in the world of software development in 2025. Safari shipped a reasonable implementation of text-wrap: pretty: https://webkit.org/blog/16547/better-typography-with-text-wrap-pretty/. We are getting closer and closer to the cutting-edge XV-century technology. Beautiful paragraphs!

We are not quite there yet, hence the present bug report.

A naive way to break text into lines to form a paragraph of a given width is greediness: add the next word to the current line if it fits, otherwise start a new line. The result is unlikely to be pretty — sometimes it makes sense to try to squeeze one more word on a line to make the lines more balanced overall. Johannes Gutenberg did this sort of thing manually, to produce a beautiful page above. In 1981, Knuth and Plass figured out a way to teach computers to do this, using dynamic programming, for line breaking in TeX.

Inexplicably, until 2025, browsers stuck with the naive greedy algorithm, subjecting generations of web users to ugly typography. To be fair, the problem in a browser is harder version than the one solved by Gutenberg, Plass, and Knuth. In print, the size of the page is fixed, so you can compute optimal line breaking once, offline. In the web context, the window width is arbitrary and even changes dynamically, so the line-breaking has to be “online”. On the other hand, XXI century browsers have a bit more compute resources than we had in 1980 or even 1450!

Making lines approximately equal in terms of number of characters is only half-way through towards a beautiful paragraph. No matter how you try, the length won’t be exactly the same, so, if you want both the left and the right edges of the page to be aligned, you also need to fudge the spaces between the words a bit. In CSS, text-wrap: pretty asks the browser to select line breaks in an intelligent way to make lines roughly equal, and text-align: justify adjusts whitespace to make them equal exactly.

Although Safari is the first browser to ship a non-joke implementation of text-wrap, the combination with text-align looks ugly, as you can see in this very blog post. To pin the ugliness down, the whitespace between the words is blown out of proportion. Here’s the same justified paragraph with and without text-wrap: pretty:

The paragraph happens to look ok with greedy line-breaking. But the “smart” algorithm decides to add an entire line to it, which requires inflating all the white space proportionally. By itself, either of

p {
    text-wrap: pretty;
    text-align: justify;
}

looks alright. It’s just the combination of the two that is broken.

This behavior is a natural consequence of implementation. My understanding is that the dynamic programming scoring function aims to get each line close to the target width, and is penalized for deviations. Crucially, the actual max width of a paragraph is fixed: while a line can be arbitrary shorter, it can’t be any longer, otherwise it’ll overflow. For this reason, the dynamic programming sets the target width to be a touch narrower than the paragraph. That way, it’s possible to both under and overshoot, leading to better balance overall. As per original article:

But if you subsequently justify all the way to the red line, the systematic overshoot will manifest itself as too wide inter-word space!

WebKit devs, you are awesome for shipping this feature ahead of everyone else, please fix this small wrinkle such that I can make my blog look the way I had intended all along ;-)

Programming Aphorisms

A meta programming post — looking at my thought process when coding and trying to pin down what is programming “knowledge”. Turns out, a significant fraction of that is just reducing new problems to a vocabulary of known tricks. This is a personal, descriptive post, not a prescriptive post for you.

It starts with a question posted on Ziggit. The background here is that Zig is in the process of removing ambient IO capabilities. Currently, you can access program environment from anywhere via std.process.getEnvVarOwned. In the next Zig version, you’ll have to thread std.process.Environ.Map from main down to every routine that needs access to the environment. In this user’s case, they have a readHistory function which used to look up the path to the history file in the environment, and they are wondering how to best model that in the new Zig. The options on the table are:

pub fn readHistory(
    io: std.Io,
    alloc: Allocator,
    file: std.Io.File,
) ReadHistoryError!void;

pub fn readHistory(
    io: std.Io,
    alloc: Allocator,
    maybe_environ_map: ?*std.process.Environ.Map,
) ReadHistoryError!void;

pub fn readHistory(
    io: std.Io,
    alloc: Allocator,
    maybe_absolute_path: ?[]const u8,
    maybe_environ_map: ?*std.process.Environ.Map,
) ReadHistoryError!void;

My starting point would instead be this:

pub const HistoryOptions = struct {
    file: []const u8,

    pub fn from_environment(
        environment: *const std.process.Environ.Map,
    ) HistoryOptions;
};

pub fn readHistory(
    io: std.Io,
    gpa: Allocator,
    options: HistoryOptions,
) ReadHistoryError!void;

In terms of meta programming, what I find fascinating is that this, for me, is both immediate (I don’t have to think about it), but also is clearly decomposable into multiple factoids I’ve accumulated before. Here’s a deconstruction of what I did here, the verbal “labels” I use to think about what I did, and where I had learned to do that:

First, I “raised the abstraction level” by giving it a name and a type (HistoryOptions). This is a rare transformation which I learned and named myself. Naming is important for my thinking and communicating process. “Let’s raise abstraction level” is a staple code review comment of mine.

Second, I avoided “midlayer mistake” by making sure that every aspect of options is user-configurable. Easy to do in Zig, where all fields are public. I learned about midlayer mistake from a GitHub comment by Josh Triplett.

Third, I provided a “shortcut”, the from_environment convenience function that cuts across abstraction layers. I learned the “shortcut” aphorism from Django Views — The Right Way. Germane to the present article, I read that post a decade after I had touched Django the last time. It was useless to me on the object level. On the meta level, reading the article solidified and named several programming tricks for me. See reverberations in How to Make a 💡?.

Fourth, I instinctively renamed alloc to “gpa” (in opposition to “arena”), the naming I spotted in the Zig compiler.

Fifth, I named the configuration parameter “options”, not config, props or params, a naming scheme I learned at TigerBeetle.

Sixth, I made sure that the signature follows “positional DI” scheme. Arguments that are dependencies, resources with unique types are injected positionally (and have canonical names like io or gpa). Arguments that directly vary the behavior of function (as opposed to affecting transitive callees) are passed by name, in the Options struct.

To be specific, I don’t claim that my snippet is the right way to do this! I have no idea, as I don’t have access to the full context. Rather, if I were actually solving the problem, the snippet above would be my initial starting point for further iteration.

Note that I also don’t explain why I am doing the above six things, I only name them and point at the origin. Actually explaining the why would take a blog post of its own for every one of them.

And this is I think the key property of my thought process — I have a bag of tricks, where the tricks are named. Inside my mind, this label points both to the actual trick (code to type), as well as a justification for it (in what context that would be a good trick to use).

And I use these tricks all the time, literally! Just answering in passing to a forum comment makes me grab a handful! A lot of my knowledge is structured like a book of coding aphorisms.

Meta meta — how come I have acquired all those tricks? I read voraciously, random commits, issues, jumping enthusiastically into rabbit holes and going on wiki trips. The key skill here is recognizing an aphorism once you see it. Reading Ziggit is part of trick-acquisition routine for me. Having learned the trick, I remember it, where “remembering” is an act of active recall at the opportune moment. This recall powers “horizontal gene transfer” across domains, stealing shortcuts from Django and midlayer mistake from the kernel. Did you notice that applying “horizontal gene transfer” to the domain of software engineering tacit knowledge is horizontal gene transfer? When entering a new domain, I actively seek out the missing tricks. I am relatively recent in Zig, but all the above tricks are either Zig native, or at least Zig adapted. Every once in a while, I “invent” a trick of my own. For example, “positional DI” is something I only verbalized last year. This doesn’t mean I hadn’t been doing that before, just that the activity wasn’t mentally labeled as a separate thing you can deliberately do. I had the idea, now I also have an aphorism.

CI In a Box

I wrote box, a thin wrapper around ssh for running commands on remote machines. I want a box-shaped interface for CI:

const repository = "git@forge.com/me/my-project";
const commit_sha = Deno.env["COMMIT"];

const runners = await Promise.all(
    ["windows-latest", "mac-latest", "linux-latest"]
        .map((os) => $`box create ${os}`)
);

await Promise.all(runners.map(async ($runner) => {
    await $`box run ${runner}
        git clone ${repository} .`;

    await $`box run ${runner}
        git switch --detach ${commit_sha}`;

    await $`box run ${runner}
        ./zig/download.ps1`;

    await $`box run ${runner}
        ./zig/zig build test`;
}));

That is, the controlling CI machine runs a user-supplied script, whose status code will be the ultimate result of a CI run. The script doesn’t run the project’s tests directly. Instead, it shells out to a proxy binary that forwards the command to a runner box with whichever OS, CPU, and other environment required.

The hard problems are in the ["windows-latest", "mac-latest", "linux-latest"] part:

• One of them is not UNIX.
• One of them has licensing&hardware constraints that make per-minute billed VMs tricky (but not impossible, as GitHub Actions does that).
• All of them are moving targets, and require someone to do the OS upgrade work, which might involve pointing and clicking.

CI discourse amuses me — everyone complains about bad YAML, and it is bad, but most of the YAML (and associated reproducibility and debugging problems) is avoidable. Pick an appropriate position on a dial that includes

• writing a bash script,
• writing a script in the language you already use,
• using a small build system,
• using a medium-sized one like make or zig build, or
• using a large one like nix or buck2.

What you can’t just do by writing a smidgen of text is getting the heterogeneous fleet of runners. And you need heterogeneous fleet of runners if some of the software you are building is cross-platform.

If you go that way, be mindful that

In other words, while SSH supports syntax like ssh $HOST cmd arg1 arg2, it just blindly intersperses all arguments with a space. Amusing to think that our entire cloud infrastructure is built on top of shell injection!

This, and the need to ensure no processes are left behind unintentionally after executing a remote command, means that you can’t “just” use SSH here if you are building something solid.

make.ts

Up Enter Up Up Enter Up Up Up Enter

Sounds familiar? This is how I historically have been running benchmarks and other experiments requiring a repeated sequence of commands — type them manually once, then rely on shell history (and maybe some terminal splits) for reproduction. These past few years I’ve arrived at a much better workflow pattern — make.ts. I was forced to adapt it once I started working with multiprocess applications, where manually entering commands is borderline infeasible. In retrospect, I should have adapted the workflow years earlier.

function $(literal, ...interpolated) {
  console.log({ literal, interpolated });
}

const dir = "hello, world";
$`ls ${dir}`;

{
    literal: [ "ls ", "" ],
    interpolated: [ "hello, world" ]
}

await Promise.all([
    $`sleep 5`,
    $`sleep 10`,
]);

#!/usr/bin/env -S deno run --allow-all
import $ from "jsr:@david/dax@0.44.2";

await $`./zig/zig build -Drelease -Dtarget=x86_64-linux`;
await $`box sync 0-5 ./tigerbeetle`;
await $`box run 0-5
    ./tigerbeetle format --cluster=0 --replica-count=6 --replica=?? 0_??.tigerbeetle`;
await $`box run 0-5
    ./tigerbeetle start --addresses=?0-5? 0_??.tigerbeetle`;

await $`./zig/zig build -Drelease -Dtarget=x86_64-linux`;
await $`box sync 0-5 ./tigerbeetle`;

await $`box run 0-5 rm 0_??.tigerbeetle`.noThrow();
await $`box run 0-5 pkill tigerbeetle`.noThrow();

await $`box run 0-5
    ./tigerbeetle format --cluster=0 --replica-count=6 --replica=?? 0_??.tigerbeetle`;
await $`box run 0-5
    ./tigerbeetle start --addresses=?0-5? 0_??.tigerbeetle`;

await Promise.all([
    $`box run 0-5 ./tigerbeetle start     --addresses=?0-5? 0_??.tigerbeetle`,
    $`box run 6   ./tigerbeetle benchmark --addresses=?0-5?`,
])

const replicas = range(6).map((it) =>
    $`box run ${it}
        ./tigerbeetle start --addresses=?0-5? 0_??.tigerbeetle
        &> logs/${it}.log`
        .noThrow()
        .spawn()
);

await Promise.all([
    $`box run 6 ./tigerbeetle benchmark --addresses=?0-5?`,
    (async () => {
        await $.sleep("20s");
        console.log("REDRUM");
        await $`box run 1 pkill tigerbeetle`;
    })(),
]);

replicas.forEach((it) => it.kill());
await Promise.all(replicas);

async function benchmark(tigerbeetle: string) {
    // ...
}

const tigerbeetle = Deno.args[0]
await benchmark(tigerbeetle);

$ ./make.ts tigerbeetle-baseline
$ ./make.ts tigerbeetle

for (const attempt of [0, 1])
for (const tigerbeetle of ["baseline", "tigerbeetle"])
for (const mode of ["normal", "viewchange"]) {
    const results = $.path(
        `./results/${tigerbeetle}-${mode}-${attempt}`,
    );
    await benchmark(tigerbeetle, mode, results);
}

Considering Strictly Monotonic Time

Monotonic time is a frequently used, load bearing abstraction. Monotonicity is often enforced using the following code:

fn now(clock: *Clock) Instant {
    const t_raw = os_time_monotonic();

    const t = @max(t_raw, clock.guard);
    assert(t >= clock.guard);
    assert(t >= t_raw);

    clock.guard = t;
    return t;
}

That is, ask the OS about the current monotonic time, but don’t trust the result too much and clamp it using an in-process guard. Under normal scenarios, you can trust the OS promise of monotonicity, but, empirically, there’s a long tail of different scenarios where the promise isn’t upheld: https://github.com/rust-lang/rust/pull/56988

Today I realized that, if you are doing the above, you might as well force the time to be strictly monotonic:

const t = @max(t_raw, clock.guard + 1ns);
assert(t > clock.guard);

The benefit of strict monotonicity is that you can tighten asserts, assert(past <= present) can become assert(past < present) and that additionally catches the bug where you pass in exactly the same instance. In other words, the <= version explicitly allows either query-ing the time again, or using the old value directly.

Conversely, with strictly monotonic time, you know that if you see two numerically identical time instances, they must have been ultimately derived from the exact same call to now(). Time becomes fundamentally less ambiguous.

The constraint here is that the resolution of the time value (not the clock resolution) needs to be high enough, to make sure that repeated +1 don’t move you into the future, but nanosecond precision seems fine for that.

Vibecoding #2

I feel like I got substantial value out of Claude today, and want to document it. I am at the tail end of AI adoption, so I don’t expect to say anything particularly useful or novel. However, I am constantly complaining about the lack of boring AI posts, so it’s only proper if I write one.

const fd = open("/etc/passwd");

const fd_local = open(.host, "/etc/passwd");
const fd_cloud = open(.{.addr = "1.2.3.4"}, "/etc/passwd");

$ cd ~/p/tb/work
$ code . # hack here
$ remote-sync
$ remote-run ./zig/zig build test

# Switch to project with local modifications
$ cd ~/p/tb/work
$ git status --short
 M src/lsm/forest.zig

# Spin up 3 machines, print their IPs
$ box create 3
108.129.172.206,52.214.229.222,3.251.67.25

$ box list
0 108.129.172.206
1 52.214.229.222
2 3.251.67.25

# Move my code to remote machines
$ box sync 0,1,2

# Run pwd&ls on machine 0; now the code is there:
$ box run 0 pwd
/home/alpine/p/tb/work

$ box run 0 ls
CHANGELOG.md  LICENSE       README.md     build.zig
docs/         src/          zig/

# Setup dev env and run build on all three machines.
$ box run 0,1,2 ./zig/download.sh
Downloading Zig 0.14.1 release build...
Extracting zig-x86_64-linux-0.14.1.tar.xz...
Downloading completed (/home/alpine/p/tb/work/zig/zig)!
Enjoy!

# NB: using local commit hash here (no git _there_).
$ box run 0,1,2 \
    ./zig/zig build -Drelease -Dgit-commit=$(git rev-parse HEAD)

# ?? is replaced by machine id
$ box run 0,1,2 \
    ./zig-out/bin/tigerbeetle format \
    --cluster=0 --replica=?? --replica-count=3 \
    0_??.tigerbeetle
2026-01-20 19:30:15.947Z info(io): opening "0_0.tigerbeetle"...

# Cleanup machines (they also shutdown themselves after 8 hours)
$ box destroy 0,1,2

type CLI =
  | CLICreate
  | CLIDestroy
  | CLIList
  | CLISync

type BoxList = string[];
type CLICreate = { tag: "create"; count: number };
type CLIDestroy = { tag: "destroy"; boxes: BoxList };
type CLIList = { tag: "list" };
type CLISync = { tag: "sync"; boxes: BoxList; };

function fatal(message: string): never {
  console.error(message);
  Deno.exit(1);
}

function CLIParse(args: string[]): CLI {

}

async function main() {
  const cli = CLIParse(Deno.args);

  if (cli.tag === "create") return await mainCreate(cli.count);
  if (cli.tag === "destroy") return await mainDestroy(cli.boxes);
  ...
}

async function mainDestroy(boxes: string[]) {
  for (const box of boxes) {
    await instanceDestroy(box);
  }
}

async function instanceDestroy(id: string) {

}

// Create spot instance
const instanceMarketOptions = JSON.stringify({
  MarketType: "spot",
  SpotOptions: { InstanceInterruptionBehavior: "terminate" },
});
const tagSpecifications = JSON.stringify([
  { ResourceType: "instance", Tags: [{ Key: moniker, Value: id }] },
]);

const result = await $`aws ec2 run-instances \
  --image-id ${image} \
  --instance-type ${instanceType} \
  --key-name ${moniker} \
  --security-groups ${moniker} \
  --instance-market-options ${instanceMarketOptions} \
  --user-data ${userDataBase64} \
  --tag-specifications ${tagSpecifications} \
  --output json`.json();

const instanceId = result.Instances[0].InstanceId;

// Wait for instance to be running
await $`aws ec2 wait instance-status-ok --instance-ids ${instanceId}`;

async function main() {
  const cli = CLIParse(Deno.args);
  const instances = await instanceMap();

  if (cli.tag === "create") return await mainCreate(instances, cli.count);
  if (cli.tag === "destroy") return await mainDestroy(instances, cli.boxes);
  ...
}