How Merge Queues Keep Main Green at Scale

About Jon Block

Jon Block has spent 26 years in software engineering, nearly all of it at high-growth startups. He has served as VP of Engineering and CTO multiple times and today advises engineering organizations through his firm, Low Touch Advisor.

How Repos Sprawl at Scale

Almost every company I've been at has used GitHub to manage their repos, and they've done what I call the multiverse pattern, meaning that it's not a monorepo. It almost always starts with a small team and a single GitHub repo, maybe a PHP or Python codebase.

If the company is growing, so is the engineering team, and the codebase starts to separate into different buckets, which start evolving separately. 

Before long, you end up with hundreds of independent repositories, each on its own trajectory. By the time the company is thousands of engineers strong, potentially public, and fielding questions about SOC compliance and build reproducibility, the multiverse has become chaos. Build engineers scramble to support every team doing things their own way.

Tools like Bazel and Buck exist to bring order to large, complex build graphs, but retrofitting them onto an existing codebase is painful.

The Stale Branch Problem

There is a misconception at the heart of how most teams merge code. A developer opens a pull request. CI runs. The checks go green. They merge. Simple, right?

Not quite. That green status reflects the code tested against the version of main that existed when the branch was created, not what main looks like now. If the PR has been open for a few days, the branch is stale. The actual code mix that gets merged may be broken.

This is the core problem merge queues solve.

Merge Queues: Keeping Main Green

Rather than merging immediately on a green PR, a merge queue accepts a developer's intent to merge, combines their changes with the very latest state of main on a throwaway branch, and runs CI again against that exact code mix. Only if that passes does the merge happen.

At scale, GitHub's built-in merge queue is too slow for organizations with hundreds of engineers all trying to merge simultaneously; that's where tools like Aviator MergeQueue come in. Two concepts become critical: swim lanes, which allow PRs touching non-overlapping files to merge in parallel, and batching, which groups pending PRs together and runs CI once against the combined code mix. Between the two, high-velocity teams can maintain merge integrity without throughput becoming a bottleneck.

MergeQueue is like a seat belt

MergeQueue is not always well received by engineers because they don’t want to wait longer for their PR to merge. But the cost of waiting is far smaller than the cost of breaking main.

You're telling someone that they're going to need to wait longer for their pull request to merge. And the trade-off is that the quality of those merges will be higher, and the company will have less downtime and outages.

It's like wearing a seatbelt. Why would you not do that? It's the only responsible thing to do.

‍Anyone who is shipping something that's business critical, especially if you automatically deploy—and there are patterns out there where once you merge, your stuff could be live within seconds. That code better work, or else you're going to be disappointing many people. In terms of convincing engineers of the benefits of a merge queue, I think that's just the only responsible thing to do if you're working on something that matters.

AI and the Build Pipeline

The amount of code that's being automatically AI-generated and deployed into live environments is terrifying because it's very low quality. There is a big opportunity for the AI-generated code to be checked more deeply. 

The more interesting opportunity is using AI to reduce the operational burden of the build system itself.

I see a future where AI captures "what the team's intent is" and handles the rest. Where the developer can just focus on the features that they want, and the build and the deploy and all that other stuff is provided in a way that they don't need to understand it.