Why Oversight Without Ownership Fails

Related on The Signal Room: Privacy and AI Governance Insights with Andre Samokish. Related from HDSC: Data Governance in Healthcare: From Policy to Operational Reality.A large health system establishes an AI steering committee composed of representatives from clinical informatics, information technology, legal, and quality. The committee meets quarterly to review AI projects, evaluate vendor proposals, and discuss institutional AI strategy. Eighteen months later, a clinical AI tool produces a series of inaccurate risk scores that go undetected for several weeks. When the error is discovered, the steering committee is asked what happened. The answer is that no one on the committee was responsible for monitoring the performance of deployed AI tools. It reviewed and approved, and nothing more.

This is the structural problem at the center of AI management in most health systems. Oversight exists. Ownership does not. Committees review proposals, assess risks, and issue recommendations. When something goes wrong with a deployed AI tool, these committees are often unable to act because their charter does not include ongoing performance monitoring, remediation authority, or direct responsibility for patient safety outcomes related to AI.

The proliferation of AI committees in healthcare has been well documented. Research published in Health Affairs has examined the growth of AI oversight structures in health systems and found that the majority of large healthcare institutions have established at least one committee or task force dedicated to AI. These bodies typically include cross-functional representation and are tasked with reviewing AI use cases, evaluating vendor solutions, and providing institutional recommendations.

What most of these committees lack is operational ownership. A study from the American Hospital Association has analyzed the charters and operating structures of AI committees in health systems and found that fewer than one in five have been given responsibility for post-deployment performance monitoring, incident response, or remediation. Most committees operate as advisory bodies that recommend and review. They do not execute, enforce, or own.

This distinction matters because oversight without ownership creates an institutional gap. When an AI tool is approved and deployed, the committee that approved it typically moves on to the next review item. Monitoring falls to whichever department deployed the tool, if it falls to anyone at all. When a performance issue arises, there is no defined owner to detect it, escalate it, or resolve it.

The consequences of ownership gaps become visible when AI tools produce harm or near-miss events. Research from Brookings Institution has examined case patterns in which AI-related patient safety issues occurred at institutions with oversight committees in place. Their analysis found that in the majority of these cases, the committee had reviewed and approved the AI tool prior to deployment. The committee had not been assigned responsibility for what happened after deployment.

In these cases, the post-deployment failure was typically detected by a clinician, a quality review, or an external complaint rather than by any structured monitoring process. By the time the issue was identified, the AI tool had been producing inaccurate or inappropriate outputs for a period of weeks or months. The institutional response was reactive, disorganized, and costly.

Joint Commission has addressed this pattern directly, noting that health systems must distinguish between advisory oversight and operational ownership of AI tools. Their position is that advisory committees serve an important function in evaluating AI use cases and establishing institutional policy, and that a separate operational function is needed to manage AI tools after deployment, including performance monitoring, drift detection, and incident response.

Oversight means watching. Ownership means being answerable for results. In practice, the difference shows up in three areas.

First, oversight committees typically operate on a scheduled review cycle. They meet monthly or quarterly, review reports, and issue recommendations. Ownership requires continuous attention. An AI tool that drifts in performance does not wait for the next committee meeting. It produces harm in real time, and the response must match that pace.

Second, oversight committees lack remediation authority. When a committee identifies a concern, it issues a recommendation to the department or team responsible for the tool. Ownership requires the authority to act directly: to suspend an AI tool, to adjust its parameters, to escalate to clinical leadership, or to initiate a patient safety review.

Third, oversight committees are rarely held responsible for outcomes. If an AI tool produces harm, the committee that approved it is not typically included in the root cause analysis or held to account for the failure. Ownership means that a defined individual or function is answerable for every AI tool deployed within the enterprise, from approval through retirement.

Research from KLAS has examined how health systems that have implemented operational AI ownership structures perform compared to those that rely solely on advisory committees. Their findings indicate that health systems with designated AI owners report faster detection of performance issues, more consistent remediation, and fewer patient safety events related to AI.

To begin with, AI tools used in a clinical environment need to have AI tool owners. These owners are responsible for deployment, use, and ongoing supervision of the tool, including incident response and corrections. An owner is liable for results and not just processes.

Next, there should be a functional division of the AI oversight committee, which should be directly assigned to the clinical and executive management levels. This unit is responsible for the AI enterprise inventory and post deployment surveillance and timely identification and resolution of all performance related issues.

Lastly, the health systems should be clear about the boundary of the consultancy (review and approval) and operational (deployment and management) functions. HHS has stressed that AI systems should be coupled with clear control systems, and the same holds for all technology, particularly AI systems. The advisory committee approves, operational owners manage, and all of these functions should be documented, funded and structured.

Health systems that incorporate ownership into their management systems will be able to scale their investments with minimal risk. Those that rely on advisory systems will soon realize that control is not the same as just having a committee.

Health Affairs has looked at the development of AI oversight mechanisms in health systems. AHA has looked into the charters and operating mechanisms of AI committees. Brookings Institution has researched case scenarios of AI patient safety in oversight committees. Joint Commission has discussed the difference between advisory oversight and operational ownership. KLAS has looked at health systems with operational AI ownership structures compared to advisory-only systems. HHS has focused on the institutional elements of oversight in health technology. This edition connects the operational and institutional elements in editions Z, AH, and AD of this newsletter.

Christopher Hutchins
Founder & CEO, Hutchins Data Strategy Consultants