How Intercom Built a Claude Code Platform for 400 Engineers

About Brian Scanlan

Brian is a senior principal engineer at Intercom in Dublin, helping keep things running well and making engineers more effective. He writes and talks about infrastructure, reliability, and developer experience. Recently he's been focused on how they use Claude Code across engineering at Intercom.

He shared in detail how they use Claude Code at Intercom in this blog post.

Platform Engineering at Intercom

 I've been working for Intercom for nearly 12 years. My role is on our platform team, and we take care of everything from our availability, cost management, and performance to internal developer productivity. 

 We've been very forward-thinking on AI, not just in terms of our product. We've been adopting the use of AI inside our software development lifecycle and putting it to use well beyond software. Almost everybody in Intercom is now using Claude Code. All I do these days is enable people using AI.

 
We've got between 300 and 400 engineers. Engineering is the biggest team in the company. R&D is a huge part of what Intercom does. The platform group is about 20 to 30 engineers. We've got engineering managers and technical program managers as well to assist us in running areas like availability, costs, and performance.

Going All In on AI

Over the last year, we put together a new full-time team and staffed it with our most senior, impactful, tenured people.

‍We decided that the transition to AI for all technical work is such a big deal that we need to put many of our highest-impact people on that. People who can get stuff done at Intercom and make decisions.

 We found that we have to have full-time attention to be able to give the right level of support and enablement and troubleshooting and tweaking. I only see us adding more people to the full-time AI team.

Driving Adoption with Sticks and Carrots

We've done some pretty direct and successful ways of getting people to use these things. About a year ago, we were talking about the adoption of AI, and we were dissatisfied. We weren't seeing transformation inside our software teams. There were some good signs; there were some areas where the tools seemed to suit certain types of work. But it was extremely poorly distributed. Some individuals were doing well, but we just weren't seeing a big transformation.

 So we set a goal. Our CTO set a goal of doubling throughput on our team — the number of pull requests per head in R&D. We could argue all day about whether pull requests are a great metric. There are loads of reasons why you could say it's not a great metric. But all of that can be true, and I think it's still completely reasonable and rational to expect a large increase in the sheer volume of work going through the system, of changes going through the system.

 If you truly believe in the impact and potential of AI and where it's going, everyone will be able to get a lot more work done at a higher quality and higher volume, and you should naturally see a large increase in the amount of throughput going through our systems. Arguably 2x isn't even big enough. Maybe it should have been 10x. But it was a good start, and you have to start somewhere.

We updated job descriptions. We made it apparent that everyone's role involves the adoption and rollout of AI in their jobs. And if you're not contributing to that, if you're not doing that, you're no longer meeting expectations. That's more stick than carrot. But you need to be extremely clear.

The more positive side: we did a lot of enablement. We ran hackathons. We've run AI immersion days. We've got Slack channels where people share success stories, ask questions, and unblock each other. We build a culture of people sharing stuff that's working and building hype about it.

AI Adoption is an Organizational Change

It's organizational change. We are demanding, effectively, that people entirely change their jobs, change how they work, and change how they produce things. To do this, you have to give room for growth. You have to provide support. You have to provide access to the tools. But you have to do things like promote people for contributing positively or give spot bonuses or publicly praise people for contributing to these things.

It's not like we would like people to adopt AI. It's like, no, we're making it the top priority, and we're going to put our money where our mouth is, and we're going to reward people who make great contributions. You've got to deal with all the change and give people support.

‍People need to be in psychologically safe environments to be able to adapt, to change things, and to fail. All of this stuff is part of the journey. But we're on the other side of this now; we've got momentum, people can see the results, and that's starting to make things easier for us.

The Skills Framework

We've been having a lot of success with skills and organizing them as a foundational building block for sharing things that work, getting feedback, improving, and really as the core part of the platform. 

At this point, we've got over 300 unique skills in a single marketplace. Teams have their own plugins, and they use this single marketplace to distribute them. But we do have a bunch of core skills set up so that we have the behavior of Claude Code well-tuned for our environment.

 A great example is our base plugin. We force it to be installed in every single installation of Claude Code. What this does is it gathers basic telemetry. It publishes, using hooks in Claude Code, event information to Honeycomb: which skills were invoked, what models are being used, and basic metadata about session information. This is a public, internally accessible data set. Anyone can use it to browse and see what's going on.

 We also collect the full session transcripts. We publish 100% of the details of every single session that people are using inside of Intercom. This is for doing deeper data mining. We do some basic tagging and indexing. Having the session data available is completely critical. It's not just good enough to have high-level metadata and observability; you need to be able to debug deeply and have all of those things ready to go without having to ask people to upload their session data.

That's been a huge unblock. But even more importantly, it gives us insights to dig into: is this skill being used? And if so, is it getting people to great outcomes fast? We want to know where skills are going wrong, or maybe we have the wrong skills, or we have gaps in coverage.

Building World-Class Skills

We have a shared set of core skills aimed at all developers inside of Intercom. We have a very high bar for these. Absolutely everything in this core set of skills must have evals with them. They must have real usage. Our philosophy is that we believe we can build enough core skills that will encapsulate or be able to do the vast majority of work that a senior engineer can do.

 To do this, we have to do things to a very high standard. It's easy to write a skill or write some automation that looks like it's mostly doing the right thing. You can use it; it might get you 50% of the way there, or maybe it might be helpful. But we can't tolerate that. 

Our ambition is making all technical work agent-first. What this means in practice is we can't afford to have low-quality skills or skills that aren't trustable. If people invoke them or if Claude invokes them, they need to immediately get on the right path to carry out the job or task.

 Monolithic skills — skills that try to do multiple things — they're just hard to test. It's like code. You need something that has a unit so that you can apply a unit test to. What this means in the world of skills for Claude Code is that we want to pick discrete individual tasks, get a skill, work with that skill, and get it to world class — as good as our best engineers at doing whatever task it is. Then lock in the behavior with evals and testing and a lot of real-world usage. Then keep it there at this high-quality bar forever. Examples of skills in these areas are like troubleshooting an issue, or fixing a flaky spec, or dealing with certain types of alarms. It's far more profound or larger in scope than just writing code.

Self-Improving Systems

We like the idea of self-updating code bases or self-improving code bases, and skills are where we started off with that. I have a weekly job that runs on all of our main GitHub repositories. It verifies that everything in the CLAUDE.md files and every file that's referred to in them, and it just fact-checks them and makes sure that any references are still accurate. The job also does a check of what has changed in the last week, any big architecture patterns or anything, and tries to encode that into the CLAUDE.md files as well.

All of our skills tend to mention: if you learn new information, you should update the skill as well. That's a good way of hinting towards the agents and people using the skills that they should be updated if the skill doesn't get things right perfectly the first time.

 Knowledge harvesting currently is pretty manual. You have to remember to tell Claude, if it gets things wrong or goes down the wrong route, why didn't this get it right the first time? And you have to remember to go in and update some guidance or update the skill. That's what we're betting on at the moment. We're pushing and asking everybody in Intercom to work like this, to know that our job and the most important thing that we're doing in R&D is making all technical work agent-first.

Owning the Platform, Enabling the Teams

So what is our role? We take strong ownership over the skills, certainly the core. We consider ourselves to be the maintainers of this. It's on us to make sure that they're great and that they're always great and that we know they're great and we can prove they're great.

But our job is not just to maintain the core; it's also to do all the great enablement work to make everyone else successful as they maintain and own their stuff. We're not trying to gatekeep skills. We're happy for everyone to have ownership of their areas and their stuff and have people decide their own quality bar in those areas as well.

 But we'll help, and we'll be proactive in providing insights or giving tips or even just answering questions. A lot of this is like we're trying to get people to change how they're working, to adopt new technologies, and to think about things a lot differently. And a lot of the work there is just having conversations, chatting with people, supporting them, and answering basic questions.

Claude Code as a Platform, not a Tool

We went all in on Claude Code, and that's been part of the success of the 2x push at Intercom. Once we decided we're on one platform, one tool, and treating Claude Code not as a tool but as a platform, that's when we saw a huge increase in use. Being clear and direct and picking one tool, we found that was critical.

 We don't even have the opinion that Claude Code is better. Any tool can be better at one period of time. But it's like arguing which cloud provider is better. It doesn't matter. You need to pick one and get all the compounding benefits of being all in on one system.