Generative AI is rapidly finding its way into accounting, auditing, finance, compliance, and operational processes. As organizations begin integrating these tools into day-to-day activities, a familiar question emerges:
How do we maintain effective internal controls when the system generating information can produce different answers to the same question?
While reading the June 2026 issue of the Journal of Accountancy, I came across an article introducing COSO’s new publication, Achieving Effective Internal Control Over Generative AI. Intrigued, I reviewed the original COSO publication and found it to be a valuable resource for organizations seeking to govern AI-related risks within the familiar COSO Internal Control Framework.
Rather than creating an entirely new governance model, COSO demonstrates how the existing five components of internal control—Control Environment, Risk Assessment, Control Activities, Information & Communication, and Monitoring Activities—can be applied to generative AI use cases. The publication also highlights many of the risks auditors and control professionals have already begun discussing, including hallucinations, data leakage, bias, model drift, third-party dependencies, and “shadow AI” usage outside approved channels.
One observation stood out while reading the guidance:
The framework is strong, but many organizations will eventually need process-level controls in addition to governance-level controls.
The Missing Layer: Process-Level Validation
Most frameworks naturally focus on governance, accountability, policies, and oversight. These are essential foundations.
However, many operational risks associated with Generative AI occur at the process level, where employees interact with AI tools every day.
Consider a procurement analyst using AI to summarize vendor contracts.
Or an accountant using AI to draft financial reporting narratives.
Or an internal auditor using AI to analyze control documentation.
In these situations, the most effective control may not be another policy document. Instead, it may be a process owner who possesses knowledge that the AI does not.
One possible control strategy is what I would call:
Subject-Matter Challenge Testing
The process owner periodically asks the AI detailed questions involving business knowledge, exceptions, historical decisions, or process nuances that only experienced personnel would reasonably know.
Examples might include:
- Why was Vendor A approved despite failing the standard threshold in 2024?
- What specific inventory reserve methodology was adopted after the 2023 acquisition?
- Which controls were modified after the ERP migration and why?
The purpose is not to “trick” the AI.
The purpose is to continuously assess whether the AI’s responses remain aligned with organizational knowledge, current procedures, and actual business practices.
Results should be documented and retained as evidence of monitoring activities.
AI Controls May Need More Frequent Monitoring
Traditional internal controls are often tested quarterly or annually.
Generative AI may require a different mindset.
Unlike traditional ERP systems, AI models are probabilistic and constantly evolving. Vendor updates, model changes, retrieval sources, prompt engineering practices, and organizational data changes can all alter output quality over time. Several commentators discussing COSO’s guidance have emphasized that AI risk assessment must be continuous rather than an annual exercise.
This suggests that organizations should consider:
- Periodic prompt testing.
- Output quality reviews.
- Exception tracking.
- Documentation of AI-assisted decisions.
- Validation of critical AI-generated content before use.
- Reassessment whenever significant model or vendor changes occur.
In other words:
Controls should monitor not only the user, but also the behavior of the AI itself.
Trust, but Verify
One phrase repeatedly came to mind while reading the COSO publication:
If an organization cannot audit its AI, it cannot fully rely on it.
Generative AI offers tremendous opportunities to improve efficiency, accelerate analysis, and support decision-making. Yet the technology’s greatest strength—its ability to generate human-like responses—can also become its greatest risk when those responses are confidently incorrect.
COSO’s new guidance provides a valuable roadmap for establishing governance and control structures around AI. The next challenge for organizations, auditors, and process owners will be translating those principles into practical, repeatable controls that operate at the process level.
Frameworks establish the destination.
Process controls determine whether we actually arrive there.
As generative AI becomes embedded within business processes, effective internal control may increasingly depend on a combination of both.
— Linden Lake
Leave a Reply