AI-powered software development has changed how software is written, tested, and shipped. Tools like GitHub Copilot, Cursor, Claude, Gemini, Tabnine, and CodeWhisperer can now suggest functions, refactor messy code, or explain APIs in plain English. For developers, this feels like pair-programming with a tireless (but occasionally overconfident) partner.

But the real question is: do these assistants actually make us more productive?

Some engineers claim they can’t imagine coding without AI anymore. Others complain about “AI slop”, code that looks neat but adds bugs or debt. Recent studies, articles, and community discussions show that the answer is nuanced.

What the Studies Say

Research results so far are mixed:

• MIT & Stanford Copilot experiments (paper) found developers solved tasks up to 55% faster with Copilot. The biggest improvements were in writing boilerplate code or working with new APIs, classic “inner loop” work.
  
• A 2025 METR study (link) reported the opposite: developers were actually less productive when using AI in open-source workflows. Important caveat: the sample size was small, and most developers were unfamiliar with the tools. Still, it shows that AI adoption isn’t a guaranteed win.
  
• Research by Stanford University echoed this nuance. Gains depend heavily on context: simple tasks benefit, but complex systems and legacy code can drag teams into extra review cycles.
  

The takeaway is clear: AI boosts productivity in some scenarios but slows things down in others, especially when the team is still learning how to use it.

What do Engineers and Engineering Leaders Say

While research provides controlled results, real-world developer communities reveal how AI tools actually play out in day-to-day engineering. One example is The Hangar DX, a curated community for senior DevOps and software engineers hosted by Aviator. It’s a space where professionals from leading companies, including Netflix, LinkedIn, Stripe, MongoDB, Discord, Docker, Red Hat, and many others, gather to share hard-earned lessons on developer productivity and platform engineering.

During a recent session on AI adoption, engineers, PMs, and dev tools experts compared notes on their experiences.

General Sentiment

Many members were experimenting with AI assistants for the first time or running small internal pilots. When it comes to trying out AI coding tools, the community approaches them with open minds, but also with caution, wanting to see real value before fully embracing them.

Tools and Adoption

The most commonly discussed tools were Cursor, Claude 3.7, Gemini 2.5 Pro, Qodo, and CodeRabbit. Teams are experimenting with these in very different ways.

Some allow AI to freely explore the entire monorepo on demand, a pattern that’s especially common with Cursor and Claude. Others take a more controlled approach, building pipelines that feed AI structured context through MCP (Model Context Protocol), often using carefully maintained “golden repos” as the source of truth.

Metrics and Evaluation

Teams are moving beyond lines of code to richer metrics:

• Daily active users (DAUs)
  
• Sessions per user per day
  
• Acceptance rate of AI suggestions
  
• Code persistence (% of generated code retained after review)
  
• % of AI-generated code merged into production
  
• Tokens consumed per developer
  
• Developer satisfaction (via Slack polls and surveys)
  

This mix gives a fuller picture of adoption, usefulness, and trust.

Strengths

AI was seen as especially helpful for:

• Prototyping quickly
  
• Summarizing failing test logs
  
• Exploring unfamiliar APIs
  
• Automating repetitive scaffolding or wiring code
  

Challenges

But teams also ran into consistent issues:

• AI struggles with legacy codebases and large polyrepos.
  
• It sometimes hallucinates or ignores clear instructions.
  
• It often suggests code that looks neat but creates “tech debt on arrival.”
  
• Evaluating whether AI output is genuinely good remains tricky.

The “Vibe Coding” Debate

“Vibe coding” is the practice of prompting AI to generate large chunks of code from vague prompts, was one of the most debated topics. Fans see it as a quick way to prototype or validate ideas, helping teams move from concept to working code in minutes.

Skeptics, however, argue that vibe coding is risky. They see it as a recipe for unreviewable and potentially irresponsible code, where speed comes at the expense of quality and long-term maintainability.

Most participants agreed that vibe coding can work, but only if guardrails are in place. These include keeping pull requests small and reviewable, ensuring strong test coverage (some suggested mutation testing), and maintaining clear boundaries between modules so AI-generated code doesn’t sprawl uncontrollably.

This perspective matches Builder.io’s analysis: vibe coding can feel magical in the short term, but without discipline, it quickly becomes a liability.

Architecture Considerations

In the Hangar DX community discussion, teams emphasized that architecture plays a big role in how effective AI coding assistants can be. Strong abstraction boundaries, for example, enforced DAGs in Bazel, help contain AI-generated changes so they don’t spread unpredictably across the codebase.

Another approach is using MCP (Model Context Protocol) to give AI a structured, scoped context, rather than letting it guess its way through massive repositories. Some groups are even experimenting with auto-evaluating changes at the build graph node level, allowing AI-generated patches to be validated in isolation before being integrated.

The key takeaway is simple: the cleaner and more structured your architecture, the safer it is to bring AI into your development workflow.

Learning Curve and Onboarding

A recurring theme in AI adoption is that these tools take time to master. Developers consistently report a steep learning curve, with productivity often dipping before it improves. Early users may find themselves slowing down as they learn how to prompt effectively and interpret AI suggestions.

To ease this process, some teams use prompt tuning and shared onboarding configurations for tools like Copilot or Cursor. Others experiment with collaborative approaches such as “pairing with AI” or even “trio programming,” where two humans work alongside an AI assistant. This setup helps new developers learn how to use AI effectively while keeping human oversight firmly in place.

Security, Reviews, and the Future

Security and review practices are evolving alongside AI adoption. Some companies have started self-hosting open-source models on-premises to maintain stricter control over sensitive code and data.

AI-assisted code reviews are also in early trials with tools like CodeRabbit, Claude, and Copilot. Developers describe these reviews as feeling more like enhanced linters; they deliver quick, shallow feedback but can’t yet replace the depth and judgment of a human reviewer.

Looking ahead, widespread adoption will depend on trust and perceived value. If AI consistently surfaces real issues, teams will embrace it; if not, developers will simply ignore its feedback. Most of the Hangar DX community members agreed that the technology will improve over time, but careful evaluation and ongoing trust-building are essential for it to become a reliable part of the workflow.

Measuring Productivity the Right Way

How should teams measure AI’s impact? Counting lines of code isn’t enough. Better approaches include:

• Commit and review activity. GitHub found pull requests rose 10–11% after Copilot adoption, suggesting faster collaboration.
  
• Code persistence. Tracking how much AI-generated code survives review is a direct measure of usefulness.
  
• The SPACE framework. Look at:
  
  • Satisfaction: Are developers happier, less frustrated?
    
  • Performance: Did quality improve, or did defects drop?
    
  • Activity: Are more tests/docs being written?
    
  • Communication: Are reviews faster and smoother?
    
  • Efficiency: Did time-to-market improve?
    
• Surveys. Adevinta, for example, surveyed engineers about Copilot’s ease of use and impact, surfacing insights that raw metrics couldn’t capture.

So, Do Coding Assistants Actually Make Developers More Productive?

The honest answer is: sometimes. Coding assistants excel at inner-loop tasks like scaffolding, prototyping, and handling repetitive code. These are the areas where they can genuinely save time and reduce mental load for developers.

Where they fall short is in dealing with legacy systems, fixing complex bugs, or working on large-scale design problems. In these contexts, human judgment and deep context matter far more than raw speed.

Teams that benefit the most are the ones that set clear metrics, invest in training, and enforce guardrails. Without these, productivity gains can quickly turn into technical debt.

The best way to think about coding assistants is like interns: they’re fast, eager, and sometimes brilliant, but they still need supervision, structure, and guidance to truly add value.

Making AI Coding Assistants Work in Practice

AI coding assistants aren’t magic productivity boosters, but they can be powerful tools when used correctly. The evidence so far shows they can speed up development, though not universally and not without trade-offs.

The teams that succeed with these tools are the ones that measure outcomes with the right metrics, provide proper onboarding and training, and build guardrails into their architecture and review processes. They also strike a balance between automation and human oversight, ensuring that AI remains an aid rather than a crutch.

Used well, coding assistants free developers to spend more time on design and problem-solving. Used carelessly, they risk burying teams in technical debt. Like any tool, the real impact depends on how thoughtfully it is wielded.

FAQs