Managing PR queues for a 100+ developer team
Ryan King
Ryan King is a Principal Engineer at Color Genomics. Previously Ryan was leading infra at Twitter and Chan Zuckerberg Initiative.
Personal mandate: Keep builds stable, improve team velocity.
Challenge
A growing developer team mixed with working on a monorepo meant regular issues where two developers would make overlapping, concurrent changes to parts of the code. These PRs would eventually get merged, causing a conflict.
“Around the time where we had about 40 or 50 developers all working in the same repo, we would regularly have cases where two developers would make changes to overlapping parts of code concurrently. Their PRs would get separately tested and reviewed, and then they'd both get merged and cause a conflict.”
Ryan King
“Merge conflicts happened around 40 or 50 developers for us. We did some work to kind of get by without Aviator, but I probably would have taken it at 50 developers in the same repo. Literally the day I was like going to start announcing it, we had three conflicts on the same day, where people had merged these problems. It became much easier for me to explain why we need this merge queuing software.”
Ryan King
Integration
The team realized that more developers meant additional merge conflicts down the line. And while they could patch together a solution to handle PR conflicts, it wasn’t enough to get by.
- 50+ developers merging meant multiple conflicts per day.
- Needed to keep mainline green while merging 100+ PRs a day
- Tried solving in-house, but didn’t compare to a managed queue.
Impact
Since onboarding with Aviator, master branch has remained green 95% of the time. The automation it provides allows us to keep everything stable and therefore keep everyone productive.
“Aviator is a very important part of our software delivery platform. We have a large and growing engineering team working in a large monorepo, and Aviator's merge queue has been extremely useful for us to keep our build green as we're merging dozens, if not approaching sometimes 100 PRs a day.
On build health, like how often our master branch is green has gone from somewhere in the high 80 percent most of the time to mid to high 90 percent of the time. That's somewhat of a proxy for developer productivity, because if that's broken, you can't merge any code.”
Ryan King
"Thankfully, I've used a few other merge queuing systems before, so I was familiar with the concepts, but the docs were enough for us to get up and running and get the initial integration going. We had some questions after that, but we were able to get it 89 percent done ourselves, and then work with the aviator team to refine and fix our configuration over time."
Ryan King
Feature highlight
Aviator’s documentation was able to help the Color team quickly onboard and pilot MergeQueue for their usecase.
