Council Post: The Investigation Economy: AI Is Making Software Cheaper To Build And Harder To Explain

Sanjay Gidwani is CEO & Founder of KOSMOS.

getty

​A change went into production Tuesday afternoon. It took about an hour to build. Then, by Wednesday, the tickets started coming in. Now, it’s Friday, and nobody can say for certain if those two things are related.

I’ve watched some version of this play out at company after company this year, and it keeps bothering me. We’ve gotten remarkably good at producing change, but we’re no better than we were a decade ago at explaining it afterward.

The Hidden Cost Of Faster Development

Start with what actually got cheap because the claim gets overstated constantly. Agentic tools have collapsed the cost of producing a working change. Tasks that once consumed much of a sprint can increasingly be completed in an afternoon, allowing smaller teams to produce significantly more change.

None of that touched the cost of understanding a change that misbehaves. That number hasn’t moved nearly as fast. Two curves moving at completely different speeds, and almost everyone is watching the one that looks like progress.

The volume argument comes next, and it’s the one I made first when I started thinking about this. More releases mean more surface area, which leads to more things that can break. If the error rate stays the same, investigation volume climbs right alongside deployment frequency. The teams absorbing that work aren’t necessarily growing along with it.

However, I’ll give up the objection before someone hands it to me. The tools writing the code are also writing the tests. Defect rates may well improve. If they improve enough, this argument gets a lot softer. Volume was never the part that should worry you.

When There’s Nobody To Call

Investigations have always had a shortcut, and anyone who has run one knows exactly what it is. You find the person who made the change and ask them what they were going for.

That conversation beats any log file. It collapses hours of reconstruction into about five minutes because the person is carrying something that never made it into a system. They know what they were solving for and which approach they threw out along the way. Usually, they know precisely which part of it they were nervous about when it shipped.

Most operators think of that as knowing who to call. When an agent writes the change, there’s nobody to call.

You still have the commit and the diff. What you may not have is any durable record of intent. An engineer may have reviewed and approved the pull request without carrying the same context they would’ve developed by building the change themselves. Six weeks later, when that integration acts up again, the institutional memory that used to shorten the investigation may not be there.

That’s the part I’d worry about; every one of these investigations could now start with nobody to ask. This stays invisible longer than it should, for reasons that have nothing to do with anyone’s competence.

Investigation time doesn’t have its own clock. It’s hidden inside the resolution window, an unmeasured slice of the total time. Your service level reports show you the whole span, but they never tell you how much of that time was just spent figuring out what actually happened before anyone could fix it.

Escalations get counted, which feels like the same thing but isn’t. True investigations, where somebody sits down and rebuilds a timeline across four systems, are a much smaller subset that almost nobody separates out.

Then there’s recurrence, which arrives looking like new work. A problem your team solved in March comes back in September under a different case number, and nothing in the queue flags it as something you’ve seen before.

Put everything together, and you have what I think of as the investigation economy: the growing amount of engineering time and organizational effort spent reconstructing what happened, why it happened and whether you’ve seen it before. It’s an operating expense with no line item, growing quietly in exactly the conditions we’re now creating on purpose.

Start Measuring The Investigation

If you run technology for a company that’s about to ship a great deal more software than it used to, the instrumentation matters more than the strategy.

Start by counting escalations and investigations separately. The second number is smaller and far more expensive, and the ratio between them may surprise you. Pull time to root cause out of time to resolution. Track only the second one, and you can spend a year improving it while the first holds completely flat. Your metric will tell you things are going well.

Look honestly at how much of your volume is work you’ve already done once.​ Then ask the uncomfortable question: When somebody finally figures out why an incident happened, where does that understanding go? If the honest answer is a ticket comment, a chat thread or one person’s head, you’re running an organization that can’t compound what it learns.

That was survivable when change was expensive and slow. It won’t survive the volume that’s coming. The constraint is moving.

For 20 years, it sat on the production side, and that’s where the tooling money went. We’re watching that side get solved in front of us. What’s left over is the side nobody bothered to instrument: reconstructing what happened, tying it to what changed and keeping the answer somewhere the next person can actually find it.

The companies that win the next decade will be the ones that can explain what happened when code breaks.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?