: the Open Questions Defining Business Next
AI agents are moving from software feature to operating-model choice. The decisive questions now concern accountability, workflow redesign, economics, security, labor, and where organizations should preserve human judgment.
Priya RamanathanFounding film criticFirst published 9/1/2026 · last revised 9/2/2026 with fresh sources, corrections, and new context. Reader corrections are reviewed and folded into future versions.
Summary
The next era of business will not be defined simply by who buys the most AI. It will be defined by which organizations can assign useful work to agents without losing control of cost, evidence, customer trust, or legal accountability. For operators, the unresolved questions are practical: which workflows should be delegated, how autonomy should be bounded, when humans must intervene, and whether productivity gains become margin, growth, or merely more output. The winners are likely to treat agents as a new operating layer—measured, governed, and redesigned around outcomes—rather than as chatbots attached to old processes.
Key takeaways
- Workflow quality matters more than model novelty: automating a broken process usually accelerates defects and exceptions.
- The durable unit of value is a completed, verifiable business outcome—not a prompt, token, seat, or generated draft.
- Agent autonomy should rise only as reversibility, observability, permissions, and evidence quality improve.
- Human review is not one control; it must be placed at specific decision gates where errors become costly or irreversible.
- ROI depends on adoption, cycle-time reduction, quality, rework, and capacity redeployment—not labor hours theoretically saved.
- Security architecture must assume agents can be manipulated through instructions, documents, tools, and connected systems.
- Buying one platform may simplify governance, while a composable stack can offer flexibility; neither choice removes integration work.
- Leadership must decide who owns an agent's actions before the agent is allowed to affect customers, money, records, or regulated decisions.
Explain like I'm 5
Imagine hiring a very fast junior colleague who can read, write, search approved systems, and use software—but sometimes misunderstands instructions and may confidently make mistakes. You would not hand that colleague the company bank account on day one. You would start with a narrow job, provide examples, limit access, check the work, and expand responsibility only after measuring results. An AI agent works similarly, except it can operate at software speed and serve many people at once. The important business question is therefore not, ‘Can the AI do something impressive?’ It is, ‘Can this system repeatedly complete a valuable task, show what it did, protect sensitive information, and know when to ask a person?’ Business next will be shaped by how well companies answer that question across sales, service, finance, procurement, operations, and management.
Deep dive
From copilots to accountable work
Generative AI first entered many companies as a writing and search assistant. Agents raise the stakes because they can plan steps, call tools, update systems, and trigger downstream work. A sales agent might research an account, draft outreach, log activity in Salesforce, and schedule a follow-up. A procurement agent might compare bids and prepare a purchase request. The defining question is where assistance ends and delegated authority begins. Operators need an explicit autonomy ladder: recommend, draft, act with approval, act within limits, or act independently. Each level should specify permitted tools, data boundaries, approval points, rollback procedures, and an accountable business owner.
Will companies redesign work—or automate its paperwork?
The largest gains rarely come from reproducing every existing handoff. A workflow diagnosis should map the trigger, desired outcome, systems touched, decision rules, exception rate, evidence required, and cost of error. Consider a lead-response process: an agent that writes emails faster adds little if routing rules are poor, records are incomplete, and sales representatives ignore alerts. Redesign might instead unify enrichment, qualification, assignment, outreach, and CRM hygiene around one service-level objective. The open management question is whether leaders will remove obsolete steps and decision rights or merely place AI on top of organizational debt.
What is the real economic unit?
Per-seat pricing fits software used directly by employees; agents complicate that model because one system can execute thousands of tasks. Token expenditure is measurable but poorly aligned with value. Outcome pricing—per resolved case, qualified meeting, reconciled invoice, or completed review—better reflects benefit, yet creates disputes about attribution and quality. A defensible ROI model begins with a baseline: volume, labor minutes, wait time, error rate, rework, conversion, and escalation. It then subtracts model, platform, integration, supervision, security, and change-management costs. Claimed hours saved count only when capacity is redeployed, service improves, revenue rises, or headcount growth is avoided.
Can autonomy coexist with security and compliance?
Agents expand the attack surface. Prompt injection may arrive through an email, webpage, support ticket, or document the agent reads. Excessive permissions can turn a model error into deleted records, unauthorized disclosure, or fraudulent payment. Controls should include least-privilege identities, tool allowlists, data classification, isolated execution, transaction limits, immutable logs, testing, and rapid revocation. High-impact decisions also require policy-specific safeguards. The EU AI Act entered into force on August 1, 2024, with obligations applying in stages; its risk-based framework makes use case and role classification important. In the United States, sectoral rules and existing consumer-protection, employment, privacy, and discrimination law remain relevant even without one comprehensive federal AI statute.
Who is accountable when an agent is wrong?
A vendor may supply the model, an integrator may configure orchestration, IT may grant access, and a business team may define the objective. None of that answers who owns a bad customer promise or an incorrect financial posting. Every production agent needs a named process owner and technical owner, plus clear incident, appeal, and escalation paths. Logs must preserve the model and policy version, retrieved evidence, tool calls, approvals, outputs, and final disposition. For sensitive workflows, the ability to reconstruct a decision is as important as average accuracy. Boards should ask not whether humans are ‘in the loop,’ but which human can stop the system and on what signal.
How will labor and management change?
Agents can compress coordination work: gathering status, preparing analysis, updating records, and chasing routine exceptions. That may widen managerial spans, change entry-level roles, and make process knowledge more valuable than raw document production. It may also create hidden review labor if outputs are unreliable. Companies should measure augmentation and displacement separately and invest in employees who can define acceptance criteria, inspect evidence, handle exceptions, and improve workflows. The central talent question is not whether every worker becomes a prompt engineer. It is whether managers can translate strategic intent into operating rules that people and machines can execute together.
Glossary
- AI agent
- A software system that uses a model to pursue a goal, select steps, and interact with tools or data under defined constraints.
- Agentic workflow
- A process combining model reasoning, deterministic rules, tool calls, state, and human approvals to complete business work.
- Autonomy envelope
- The explicit boundary of actions, systems, values, time periods, and exceptions within which an agent may operate.
- Human-in-the-loop
- A design in which a person reviews or authorizes specified decisions; useful only when the intervention point and response time are defined.
- Retrieval-augmented generation
- A method that supplies a model with selected external documents or records so its answer can use current, organization-specific context.
- Prompt injection
- Malicious or conflicting instructions placed in content an AI system processes, potentially causing it to ignore intended rules or misuse tools.
- Evaluation
- A repeatable test of task success, factuality, policy compliance, tool behavior, latency, cost, or other production criteria.
- Observability
- The ability to inspect an agent's inputs, intermediate actions, tool calls, outputs, costs, failures, and version history.
- Process owner
- The business leader accountable for a workflow's outcome, controls, exceptions, and performance—even when automation performs the steps.
FAQs
Where should a company deploy its first AI agent?+
Choose a frequent, bounded workflow with digital inputs, measurable outcomes, and reversible actions. Good candidates often include research briefs, ticket triage, meeting preparation, CRM cleanup, or invoice exception preparation rather than autonomous payments or employment decisions.
How is an agent different from ordinary automation?+
Traditional automation follows predefined paths; an agent can interpret unstructured context and select among tools or steps. Strong systems combine both: models handle ambiguity while deterministic code enforces calculations, permissions, and policy.
What should an AI-agent business case include?+
Record baseline volume, handling time, waiting time, error and rework rates, conversion or service outcomes, and current cost. Add implementation, inference, software, integration, review, security, training, and maintenance costs, then value only benefits that can be realized operationally.
Should every agent require human approval?+
No, but approval should reflect consequence and reversibility. Low-risk drafting may need sampling, while customer commitments, payments, account changes, regulated advice, and personnel decisions usually warrant stronger gates or strict transaction limits.
Can agents use confidential company data safely?+
They can be designed to do so, but safety is not automatic. Buyers should verify data retention, training use, encryption, identity controls, regional processing, subcontractors, deletion, access logging, and contractual incident obligations.
How should quality be measured?+
Use task-specific acceptance criteria rather than one generic accuracy score. Track outcome success, unsupported claims, policy violations, exception and escalation rates, human corrections, latency, cost, and performance by customer or case segment.
Build, buy, or combine?+
Most organizations will combine vendor models and platforms with proprietary workflow logic, integrations, policies, and evaluations. Build more where the workflow differentiates the business or demands unusual control; buy more where the process is standard and vendor controls are adequate.
Who should own an agent program?+
A cross-functional group should set standards, but each agent needs a business process owner. Technology, security, legal, risk, data, procurement, and affected employees should participate according to the workflow's consequences.
Sources & references
- NIST AI Risk Management Framework (AI RMF 1.0)
- NIST Artificial Intelligence Risk Management Framework: Generative AI Profile
- Regulation (EU) 2024/1689 — Artificial Intelligence Act
- OWASP Top 10 for Large Language Model Applications
- Executive Order 14110 on Safe, Secure, and Trustworthy Artificial Intelligence
- The 2025 AI Index Report — Stanford Institute for Human-Centered AI
- Attention Is All You Need
- Model Context Protocol — Introduction
| Suite-native agent | Composable agent stack | Managed outcome service | |
|---|---|---|---|
| Best fit | Work concentrated in one SaaS ecosystem | Differentiated cross-system workflows | Standardized process with clear deliverables |
| Initial speed | Often fastest where connectors already exist | Moderate; orchestration and integrations must be built | Fast after data access and service definition |
| Control and customization | Moderate; bounded by vendor platform | High; policies, models, tools, and evaluations can be selected | Low to moderate; provider controls implementation |
| Primary cost basis | Seat, capacity, credits, or usage | Models, infrastructure, engineering, and operations | Transaction, case, or contracted outcome |
| Main governance risk | Vendor concentration and opaque platform behavior | Fragmented ownership and integration complexity | Weak audit access and ambiguous accountability |
| Exit difficulty | Medium to high if workflows use proprietary features | Medium when interfaces and traces are portable | High if process knowledge and data remain with provider |
A practical blueprint for turning AI agents into a secure, measurable operating layer for executive decisions, sales execution, workflow diagnosis, and company-wide automation.
A boardroom-ready framework for governing AI agents across risk classification, data access, human oversight, vendor controls, testing, monitoring, and audit evidence.
A boardroom-ready framework for funding AI-agent pilots, measuring their economics, containing risk, and deciding which workflows deserve production scale.
A practical operating model for using AI agents to improve sales responsiveness, consistency, and conversion while preserving consent, judgment, security, and the human credibility behind every customer relationship.
A boardroom-ready framework for estimating AI-agent budgets, exposing workflow constraints, sequencing pilots, and setting delivery expectations that survive contact with production.
From our own rounds
Measured on Agent Oracle, from real sessions people played on this site — not a third-party dataset.
- Rounds played here
- 27
- Questions per round
- 1