Flow, AI, and the Future of Engineering with Steve Pereira

About Steve Pereira

Steve Pereira has spent over two decades improving the flow of work across organizations. He's worked through tech support, IT management, build and release engineering, and as a founding CTO for enterprise SaaS. He serves as lead consultant for Visible Value Stream Consulting, as a board advisor to the Value Stream Management Consortium, as chair of the OASIS Value Stream Management Interoperability technical committee, and as co-founder of the Flow Collective.

What Is Value Stream Mapping?

The classic sense of value stream mapping comes from manufacturing. The origin is a field of study and management science called time and motion studies: essentially, let's watch work happening and measure it so that we can find out where things are getting stuck or slowed down.

Value stream mapping really allows us to visualize a flow of work and then quantify it, measure how long it takes to do each activity, and then measure how much time passes between those activities. The most important thing about it is that it connects end to end, from idea to delivery of value all the way to the customer.

In knowledge work and digital work, we don't have assembly lines. We can't follow the bits. Consequential value streams like product development or R&D take lots of time, have many contributors, plenty of handoffs, and work is often flowing through multiple different systems of management. So it's very difficult to actually track. 

What I love about value stream mapping is that it brings together everyone who's involved in creating and delivering value so that we can have a conversation about how that's going. You could throw away the map, but the mapping process is really valuable.

How AI Is Changing the Value Stream

Early on, we had all this activity around development. You could write code faster, and that got really exciting until you realized that we have to review all that code, and it's not going anywhere because it's low quality, and we don't have enough people to test it. All the downstream requirements were not in place.

What I've seen since is more of an investment in supporting capabilities: platforms, guidelines, templates, standards, specs, and spec-driven development. Things that make the development practice more robust, more consistent, more reliable, higher quality and reduce the review and testing burden. That creates more confidence in production.

You'll hear a lot about this idea that writing lines of code was never the bottleneck. I agree with that. But there's a big difference between writing the code for a feature and writing code for all that supporting infrastructure that makes it possible to write that feature faster, more reliably, more consistently, and with higher confidence.

The most successful organizations are looking at AI and asking how they can use it to accelerate all those other bottlenecks they've had: challenges with reviews, CI/CD pipelines, environmental availability, scheduling, scaling, operations, and maintenance. They are investing even more in the capabilities they'd already significantly invested in.

Context Switching and Flow State in the Age of AI

As engineers and as leaders, there is value in intentionally exploring and saying, I'm going to run a different experiment every week. But there's a very real risk of just doing 20 random things that don't actually help you make progress or ultimately get shipped.

Context switching is potentially valuable if your intentional goal is exploration and switching contexts because those contexts have distinct value. Otherwise it's a very real risk. Each of those threads has to be maintained the same way that if you open a branch and don't merge it back, you're going to have problems.

Context switching should be measured the same way we measured it for machines. There's a cost to throwing away the context and loading in a new one.

Our ability to actually load in context has increased dramatically. I can quickly switch to a new task and say, Catch me up to speed, and that can be distilled clearly. My ability to context switch has dramatically improved, which raises another question: is that a benefit or a risk? 

If it's easier for me to act under a state of distraction, I might end up being more distracted.

It comes back to: what is your goal? What's your strategic imperative? Unless you can tie all your actions back to that, you might look back in a year, three years, five years and think: Tthat was fun, that was interesting, but could I have gotten further?

The key aspect to remember is intentionality. If you're context switching because you have external demands on your time, you're constantly interrupted by fires or stakeholders or even your direct reports; that's an unsustainable situation. That's not the type of context switching we want.

We want it to be an intentional activity that is part of exploring a new thread or shifting our focus because we have something on the right track and we have the capacity to focus elsewhere temporarily.

And we can be in a flow state running in the wrong direction. We can also be in a flow state doing very low value work. If you're doing a commoditized task manually that could be outsourced or automated, get rid of that stuff so that you can spend more time on higher-value activities.

Rising Up the Stack: Engineers as Orchestrators

There's a natural evolution through history, and it's a pattern we're seeing repeated. We are constantly building at higher and higher levels of abstraction. Before we had AWS, APIs, and Terraform, we spent a lot of time working on one machine, and every time something happened to that machine, it was like our whole day was gone.

We got really good at that. It took serious engineering expertise, effort, and context. And now we've thrown that all away. Who cares about manual configuration changes on a single machine? I don't think anybody misses it.

There are a lot of things we're going to rise up the stack with and leverage abstractions. We could lament the loss of those things, but what's going to be ahead of us in terms of what's possible is really exciting. We don't even know what that is yet.

Value Stream Mapping as a Superpower for AI Success

There is a ton of variety when it comes to value stream mapping, because ultimately what matters is having a goal, something you want to achieve. That could mean fewer production defects, fewer late nights fighting fires, more throughput or more delivery of value to customers, and faster feedback from your customers. Having that goal is critical, and we often don't step away from the work to really think about it.

You can still have incredible value from being in a room with people, putting Post-it notes on a wall. I doubt that will ever go away. That's really, really valuable. And again, you could throw the map away afterward and still have a much better shared context to work from.