The Consent Crisis

An outpatient procedure patient checks into a large health system. A registration attendant provides a general consent form, which includes a consent clause to use her health information for quality-improvement initiatives and care coordination. What she does not know is that her clinical information will be used to develop a predictive algorithm to determine which patients may be at a higher risk for post-surgical complications. She is not informed that her data will be subjected to algorithmic evaluation and the resulting risk score will impact her treatment. The algorithm is executed. A score is produced. The care team reviews the score. The patient is not aware of any of these events.

This is a completely plausible scenario. Health systems that have adopted AI technology for clinical or administrative processes have this as a standard operating procedure. Informed consent must be revisited as the regulations that underpin these legal documents must focus on treatment data collected for billing and administrative processes. There is a new layer of data use with AI technology that these documents do not cover. Automated systems have secondary uses of patient data through algorithmic clinical recommendations or risk assessments.

The Health Insurance Portability and Privacy Act (HIPAA) allows the use of protected health information (PHI) for treatment, payment, and health care operations without needing individual patient consent. This has been used by many health systems to justify the training and deploying of AI models since AI-generated treatment insights are also considered support treatment decisions. This has a legal basis, and a full ethical justification is still lacking.

Within the Department of Health and Human Services, the Office for Civil Rights has stated in published documents that HIPAA does not require patient consent to use de-identified data. However, the de-identification standard used within HIPAA has been scrutinized by researchers who have documented re-identification risks using publicly available data. For example, a study published in Nature Communications demonstrates that, for a randomly drawn sample, 99.98% of individuals can be re-identified using only 15 demographic variables, even if the dataset has been subjected to standard de-identification techniques.

Data privacy laws at the state level also create a complex legal landscape. The California Consumer Privacy Act, along with similar laws in other states, gives individuals control over how their data is collected, used, and shared, which, in certain instances, applies to health data, and imposes additional requirements past HIPAA. For health systems that operate in several states, there are multiple conflicting requirements and no single set of rules that govern the legal landscape.

The first principle for informed consent in medicine is that people have the right to make decisions about what is done to their bodies and what data about them is used for. With regard to treatment recommendations, patients have the right to understand the reasoning of the physician, what alternatives exist, and what the risks are. If a recommendation is informed by an AI model, a patient should have the same type of access, however, the consent framework within most health systems does not accommodate the use of AI.

On this point, the American Medical Association advocates that patients disclose the use of AI in their treatment and allow them to understand its role in decision-making. The Hastings Center generated the most relevant research on the disparity between the expected ethical standards of informed consent and the reality of AI in clinical settings and found that most institutions had not revised their consent process to accommodate the use of AI.

Research on patient attitudes toward the use of personal data for AI that has been published in the Journal of Medical Ethics shows that most patients are willing to have their data used for AI if they are informed and given the option, and this willingness decreases markedly if they learn that the data use occurred without their consent. The concern is not about patients opposing the use of AI, the concern is about the absence of consent from patients.

Addressing the consent gap requires changes at the policy, process, and technology levels. At the policy level, health systems need consent frameworks that explicitly cover certain AI use cases, such as the training, deploying, and risk scoring or clinical recommendations. These frameworks need to differentiate between various forms of AI and flexibly calibrate consent requirements based on data use and its impact on the level of care.

At the process level, consent needs to be embedded into clinical workflows where it is practical for clinicians, as well as for patients. Research from Stanford Medicine has studied models of dynamic consent, allowing patients to see and revise preferences for data use, controlled through digital platforms. This approach mitigates the challenge of static consent forms, which capture a singular decision at a single point in time, leaving evolving data use unaddressed.

Technologically, health systems want the ability to enforce consent preferences at the data pipeline level, and to substantiate which patient data is used in what AI model. The Office of the National Coordinator for Health IT has incorporated some of the necessary building blocks for this infrastructure in its standards on access to patient data and interoperability, yet the requirements for AI consent management remain largely unsubstantiated.

Healthcare systems face multiple pitfalls by ignoring the consent gap. As states continue to enact privacy laws and the federal government tightens regulation on health AI, the potential for privacy breaches increases. The lack of openness on data usage leads to a loss of trust from the patient. As courts begin to rule on the adequacy of consent, the legal exposure of health systems will increase.

For health system leaders, the answer to the question of consent is not about compliance. It is about whether a system truly values the autonomy of the patient or is simply paying it lip service. Systems that value autonomy will foster long-lasting partnerships, while those that fail will create irreparable damage.

Under HIPAA, PHI can be used for treatment, payment, and operations. OCR has issued updates regarding the use of de-identified data. Risks of re-identification have been reported by Nature Communications. CCPA and similar state privacy laws impose additional compliance obligations. AI and patient consent have been addressed by the AMA. The Hastings Center has reported on the lack of consent and the use of AI. Patient attitudes on the use of AI and patient data have been reported by the Journal of Medical Ethics. Stanford Medicine has studied models of dynamic consent. ONC has developed standards for the access and sharing of patient data. This issue relates to patient rights and institutional design, which can be found in Editions W, Y, and L of this newsletter.

Christopher Hutchins
Founder & CEO, Hutchins Data Strategy Consultants