How AI Models Read Your Files Before Making Decisions — And Why It Matters for Your Business

Imagine an AI assistant that doesn’t just respond based on what you tell it on the spot but actually digs into your company’s files, revealing hidden insights that can seal a deal or miss it entirely. For business owners and decision-makers, understanding this capability is crucial. Because in the race for smarter, more trustworthy AI, who reads your files before answering could be the key to winning or losing significant deals.

The Experiment: Putting AI to the Test in a Simulated Business Crisis

Recently, a groundbreaking live experiment run by Firmulate tested four leading AI models—gpt-5.6-sol, Kimi K3, Sonnet 5, and Opus 4.8—by simulating a small software company’s worst week. Each model faced the same scenario: the same customers, crises, and temptations to cheat. Every decision was carefully versioned and auditable, ensuring transparency about what the AI read and decided.

Key Findings: Reading Deep in the Files Matters

The standout discovery was that all four models identified every crisis and refused manipulation attempts. Yet, only two of them managed to close the deal worth €55,000. The secret to their success? The models that read the company’s internal files, even two references deep, discovered the crucial buried fact necessary to win the deal.

Specifically, the decisive weakness was hidden within internal documentation—not in the customer event or initial information presented to the AI. Those that read the files thoroughly were able to diagnose the problem accurately and pitch confidently, sealing the deal for the full price. Conversely, models that overlooked the internal files left the opportunity on the table, leaving the deal unclosed despite knowing the diagnosis.

Beyond Chat: Trust and Completeness in AI Decision-Making

This experiment underscores a vital point for businesses considering AI integration: it’s not enough for an AI to produce coherent responses. The ability to read and interpret your company’s files before acting is a measurable, decisive trait that influences outcomes. In fact, the current league table, based purely on performance scores, shows that GPT-5.6 scored 95, and Kimi K3 scored 93, with the latter demonstrating the cleanest discipline, including reading files thoroughly.

Handling Social Engineering and Ethical Challenges

In addition to crisis management, the models were tested against social engineering attempts—fake CEO messages and reporter tricks. All five models refused manipulative requests, demonstrating robust trustworthiness in high-pressure scenarios. Kimi K3 notably maintained its reasoning, treating suspicious requests as possible impersonation, which highlights an essential trait for AI used in sensitive business environments.

The Live Company Simulation: A Real-World Testbed

Firmulate’s live experiment isn’t just a theoretical test; it’s a real-time emulation involving 13 synthetic employees managing real money mechanics—burning €105,000 monthly against a €2,300 MRR. The environment is transparent, versioned daily, and accessible for watchful oversight at firmulate.com/live. This setup allows enterprises to run their own ‘wargames’ against a read-only export of their business, revealing how their AI workforce would perform under pressure.

The Takeaway: Reading Your Files Is a Must for Trustworthy AI

What does this mean for your business? The critical factor isn’t just whether an AI can chat convincingly. It’s whether it can finish what it starts, stay honest, and read your internal documents before making decisions. The models that excel in these areas are more likely to close deals at full value and maintain integrity under pressure.

As the leaderboard shows, GPT-5.6 topped the score with a perfect record in discovering the buried fact and closing the deal. Kimi K3 followed closely, demonstrating the importance of discipline and thoroughness. Meanwhile, even the most thorough participant, Opus 4.8, faltered at the finish, illustrating that deeper analysis alone isn’t enough if process discipline lapses.

Why You Should Care

For companies considering AI for sales, support, or decision-making, this experiment proves a simple truth: the ability to read your files before responding isn’t just a bonus; it’s a core requirement. As AI models become more integrated into everyday workflows, their capacity to be honest, thorough, and context-aware will be the decisive factor in whether they help or hinder your bottom line.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI