On November 25, 2024, Bulgaria’s Sofia District Court made a request for a preliminary ruling to the CJEU relating to the provisions on automated decision-making (“ADM”) under the AI Act. Case C-806/24 relates to a claim made by a telecoms company against a consumer who did not pay his bills. The consumer argues that the telecom company’s method of automatically calculating fees constitutes an ADM system subject to Article 86(1) of the AI Act, and raised questions about the transparency, human review, and fairness aspects of the ADM system. The Bulgarian court requests to the CJEU clarification on 17 questions of law, citing the AI Act, the Unfair Terms Directive, and the Consumer Rights Directive.
Broadly, the Bulgarian court raises three sets of questions for the CJEU to consider:
- Questions relating to the consumer’s right to be informed and to request review of decisions made by ADM;
- Questions relating to the fairness of charging a compensation fee when the contract has ended, under consumer protection law; and
- A question relating to whether a consumer may be ordered to bear part of the costs of the proceedings on the grounds of non-payment of amounts due under the contract, if the trader did not explain the ADM in a clear and comprehensible manner, but later provides the explanation during the judicial proceedings.
The first set of questions centers on the interpretation of Article 86(1) of the AI Act. This provision provides that any individual, who is subject to a decision which (1) is based on output from a high-risk AI system and (2) produces legal effects or similarly significantly affects that person, has the right to obtain from the AI deployer “clear and meaningful explanations of the role of the AI system in the decision-making procedure and the main elements of the decision taken”.
The Bulgarian court asks, among others, the following questions regarding Article 86(1) of the AI Act, read in conjunction with other EU laws:
- Should Article 86(1) be interpreted as meaning that the trader must inform the consumer how the decision using ADM is made (here, regarding what algorithm calculates the automatically generated invoices, and what elements and parameters are fed into it)?
- Does the court have the power to require the trader to provide the black box data, source code and the algorithm relating to the way decisions are made based on an ADM system?
- Does the consumer have the right under Article 86(1) to demand from the trader an explanation or the algorithm relating to the decision-making process?
- Does the consumer have the right to request human review of decisions based on ADM – and does that human review need to be by a judge in real judicial proceedings?
It is interesting to note that while the AI Act entered into force in August 2024, Article 86(1) of the AI Act does not apply until August 2, 2026. A judgment to be expected by mid-2026.
The Covington team continues to monitor regulatory developments on AI, and we regularly advise the world’s top technology companies on their most challenging regulatory and compliance issues in the EU and other major markets. If you have questions about AI regulation, or other tech regulatory matters, we are happy to assist with any queries.