Council Post: Every AI Agent Decision Requires Strong Evidence

Dr. Son Nguyen is Orient Software’s CTO & co-founded Neurond AI, companies specializing in software development, AI, & data science services

getty

​AI is no longer a side experiment in our business. It’s embedded in how we operate every day. Beyond helping clients build AI-powered systems, our developers also develop AI agents to support internal workflows.

These agents handle a wide range of tasks: from a meeting agent that captures decisions and action items to a people and culture agent that helps employees quickly resolve HR and IT-related requests. The goal is simple: allow teams to spend less time on repetitive work and more time on high-value activities.

We were proud of the progress. The outputs looked professional. The agents sounded confident. Then we caught one of them lying.

That discovery changed how we think about trust, accountability and what it really means to build production-grade AI systems.

Why 'How Do We Make AI Smarter?' Is The Wrong Question

One of our earliest projects, the AI meeting agent, aimed to save employee time by automatically generating meeting summaries, decisions and action items. At first glance, it felt like a success.

The summaries were professional and confident. But over time, we noticed the agent occasionally included decisions that had never been made or assigned action items that no one had agreed to own. Because the output sounded trustworthy, mistakes were difficult to spot.

Our first reaction was predictable. We assumed this was an intelligence problem and the solution was better prompting. We added more instructions, rules and edge-case handling.

The result was that things got worse.

The agent became slower, more expensive and less consistent. We learned that prompts are guidance, not guarantees. Reliability comes from having a clear specification and a system that verifies whether the output meets it.

More instructions do not automatically create more reliability. That lesson cost us real time to learn.

The turning point came during an internal review session, when one of our engineers asked a question that reframed everything: “How do we know the task was completed?” Not “How can we get the agent to follow instructions better?”

That question changed our entire perspective. We stopped focusing on what the agent was saying but rather how the agent could prove what it had done. Because in business, confidence isn’t evidence. A polished report or a well-written explanation isn’t evidence, either. Proof matters.

That shift, from trusting the explanation to requiring evidence, became the foundation of how we build AI systems today.

Lessons About Building Reliable AI Systems

Evidence Over Explanation

An AI agent telling you it completed a task is not proof that the task was completed.

Human teams understand this naturally. If a project manager says a task is finished, we often expect supporting documentation, deliverables or measurable outcomes.

The same principle should apply to AI.

For our meeting agent, the specification became simple: Every action item or decision must be traceable to a specific part of the meeting transcript. If the system cannot produce evidence, the item fails validation and is excluded from the final summary. The agent is no longer judged by how convincing its explanation sounds, but by whether its output satisfies the specification.

This doesn’t require a smarter model. It requires a smarter system around the model.

Giving AI Only What It Needs

We learned this lesson while improving our meeting agent.

At one point, we believed that giving the agent more context would lead to better results. We loaded it with tons of meeting guidelines, note-taking rules, formatting instructions, examples and edge-case scenarios.

We were wrong.

The more information we added, the harder it became for the agent to focus on what actually mattered: identifying decisions, action items and key discussion points from the meeting itself.

Eventually, we simplified. Instead of overwhelming the agent with extensive instructions, we provided only the information needed for the specific task at hand.

We learned that when an agent has too much information, it doesn’t always know what to prioritize. Often, giving AI less context produces better results than giving it more.

Fixing The System, Not The Output

Early in our AI journey, we treated failures the same way many organizations do. When an agent made a mistake, we fixed the output and moved on.

Now we ask a different question: What does the system need to catch this failure on its own next time?

This led us to build review and learning loops into our agents. After each run, failures are logged, analyzed and used to improve future performance.

Rather than treating mistakes as isolated incidents, treat them as opportunities to strengthen the system, so every failure becomes a learning opportunity.

How You Should Think About AI Today

Across all of our agent projects, we’ve moved away from thinking of AI agents as software features and toward thinking of it as an autonomous contributor. Like people, they need structure, feedback, accountability and measurable outcomes.

That means designing environments where success is measurable, actions are observable, results are verifiable and failures generate learning.

This framing changes how we allocate resources, too. We invest less in trying to improve raw model performance and more in building systems that make their outputs trustworthy.

Also, managing AI agents turned out to be more like managing teams than managing software. As leaders, you can’t simply trust people and hope for the best. You need review processes, checkpoints and ways to verify the work.

The teams getting the best results from AI aren’t necessarily using the best models. They're the ones with clear processes, built-in verification and a commitment to continuous improvement.

Conclusion

Working with AI agents has brought us back to principles that have always been true of building reliable systems and high-performing teams: clear expectations, proof and continuous improvement.

We stopped trying to make our AI smarter. We started making our systems more accountable. Instead of trusting what an agent said, we focused on verifying what it actually did.

As AI becomes more embedded in business operations, success will depend less on having the most advanced model and more on having processes that can verify results, catch mistakes and improve over time.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?