Keypoint: The Texas Attorney General reaches a first-of-its-kind settlement with a healthcare company that provides generative AI products.
On September 18, 2024, the Texas Attorney General announced that it had reached a settlement with a Dallas-based artificial intelligence healthcare company. The Attorney General’s press release represents that it is a first-of-its-kind settlement, resolving allegations that the company deployed its artificial intelligence (“AI”) products at Texas hospitals while making false and misleading statements about the safety of its products.
The Attorney General’s office alleges in the Agreement that the company made representations about its generative AI products that were false, misleading, or deceptive, that may have violated the Texas Deceptive Trade Practices Consumer Protection Act (DTPA). The company develops, markets, and deploys AI products for use by healthcare facilities, including hospitals. The Agreement provides that physicians and medical staff can use these products to assist them with treating patients and that at least four Texas hospitals were providing patient data through the company’s products.
While advertising and marketing its products, the company represented that its generative AI products were highly accurate. Specifically, the company’s advertising stated that its AI products had hallucination rates of less than 0.001%. The Attorney General’s investigation found that these metrics were likely inaccurate and may have deceived hospitals about the safety and accuracy of the products.
The Agreement requires the company to comply with the following provisions for five (5) years following the effective date of the Agreement:
- Clear and Conspicuous Disclosures – Marketing and Advertising: The company must clearly and conspicuously disclose the meaning of any metric, benchmark, or measurement, and the method, procedure, or process used by it to calculate the metric, benchmark, or measurement.
- Prohibitions Against Misrepresentations: The Company may not make any false, misleading, or unsubstantiated representations, whether express or implied, regarding any feature, characteristic, function, testing, or use of the products, or regarding the accuracy, functionality, purpose, or any other feature of its products. It must also disclose any financial or similar arrangements between it and any person who participates in its marketing or advertising, or endorses or promotes any of its products.
- Clear and Conspicuous Disclosures – Customers: The Company must provide to all customers documentation that clearly and conspicuously discloses any known or reasonably known harmful or potentially harmful uses or misuse of its products. The Agreement specifically identifies the type of information that must be included in the documentation.
Based on this settlement agreement, companies deploying AI products should consider the accuracy of their representations, both directly to customers and through any marketing efforts. Texas is taking a clear stance that AI companies must be transparent about the risks, limitations, and appropriate use of any AI products.
The Texas enforcement action also should be read in conjunction with recent guidance from the Federal Trade Commission (FTC). In January, the FTC published a blog reminding AI companies that they need to abide by the promises made in their privacy commitments to consumers when using AI technologies. In February, the FTC published another blog emphasizing that AI companies cannot quietly and retroactively change their terms of service and privacy policies to allow for the sharing of consumers’ data with third parties or using that data for AI training.