EU’s Highest Court Expands EU GDPR Restrictions on Automated Decision-Making, Profiling, and AI

A recent decision by the Court of Justice of the European Union will extend the EU General Data Protection Regulation’s automated decision-making restrictions to many present and future use cases of such technologies. While the case at issue concerned the use of automated credit scoring in credit applications, the restrictions may apply to other sectors and organizations using these technologies to generate outputs for a third party’s use in decision-making. In turn, certain EU member states are considering local law amendments to facilitate certain of the data processing activities that may be impacted by the decision.

Since coming into force in May 2018, the EU General Data Protection Regulation (GDPR) and UK GDPR have applied to many automated decision-making processes, artificial intelligence and machine learning (AI/ML), and profiling-related technologies (collectively, ADM). In particular, the EU GDPR and UK GDPR restrict data controllers from making a “decision” that is based “solely” on “automated” processing, including profiling, which produces “legal effects” concerning a data subject or which “significantly” affects them (the ADM restriction).

Until recently, it was understood that the ADM restriction applied to a business that performed a “solely” ADM process and subsequently made a “decision” significantly affecting a data subject. However, a recent important decision of the EU’s highest court, the Court of Justice of the European Union (CJEU), in the Schufa case [1] expands the EU GDPR’s ADM restriction to many current and future use cases of ADM technologies.

Specifically, the EU GDPR’s ADM restrictions are now potentially engaged where an ADM process is performed by a service provider and another entity relies on the output of that ADM process in arriving at a “decision” that significantly affects a data subject. Therefore, because of the Schufa decision, the EU GDPR (including its ADM restrictions) will potentially apply even when the user of an AI/ML output did not create such AI/ML output.

EU GDPR AND UK GDPR RESTRICTION ON ADM TECHNOLOGIES

The EU GDPR and UK GDPR both currently contain the ADM restriction. The ADM restriction, if applicable, can potentially impose a high bar for the lawful use of many ADM technologies subject to the EU GDPR or UK GDPR.

The ADM restriction does not apply in three scenarios, namely where the restricted processing (1) is “necessary” for entering or performing a contract; (2) is authorized by EU, EU member state, or UK law to which the controller is subject and which includes suitable safeguard measures; or (3) has been explicitly consented to by the data subject.

Importantly, however, even if ADM restrictions are not applicable, other EU GDPR and UK GDPR requirements may continue to apply, such as controllers being transparent with data subjects about personal data processing.

CJEU’S EXPANSIVE APPROACH IN SCHUFA

In Schufa, the CJEU determined that, for the purposes of the EU GDPR, the creation of a credit score relating to a data subject by a credit reference agency (CRA) constituted a “decision” for the purposes of the ADM restriction. Notably, in its consideration of whether the data subject had been significantly affected by the creation of the score, the CJEU had regard for the subsequent use to which the credit score was put by the downstream third-party lender and the fact that it played a determinative role in the lender’s credit decision.

The CJEU’s decision appears to be focused on ensuring that data subjects obtained, in the court’s view, the full protection of the EU GDPR’s ADM restriction. In turn, the court considered that the CRA’s creation of a credit score (by means of an ADM process) relating to a data subject was not merely a preparatory act that was decoupled from a subsequent credit decision relating to such data subject taken by a retail bank relying heavily on such credit score.

The CJEU’s decision also highlighted that the CRA’s ADM processes are subject to the EU GDPR’s many requirements, not just the ADM restriction. These requirements include controllers being transparent with data subjects about the use of, and consequences arising from, ADM processes.

CERTAIN KEY TAKEAWAYS OF SCHUFA

• While there has been much (justifiable) fanfare regarding the coming into force of the EU’s proposed landmark AI Act in the future, organizations may nonetheless need to consider the impact of the EU GDPR’s landmark Schufa decision, which is already applicable.
• The Schufa decision has important implications for CRAs and those who rely on CRA services for decision-making purposes. They may now wish to revisit their processes in the light of the potential applicability of the ADM restriction to a CRA’s creation of a credit score. Lenders may wish to consider the extent to which a credit score generated by an ADM process needs to be balanced with human-led oversight or intervention in the generation of the data subject’s “final” score.
• Though the CJEU in Schufa was concerned with the EU GDPR’s ADM restriction as it is applied to the use of automated credit scoring in credit applications, there is no indication that the court intended it to be an industry-specific decision. Indeed, the EU GDPR’s ADM restriction may well apply in a non-credit-related use case, e.g., in connection with a score used in a health insurance “decision.”
• More generally, the Schufa decision may apply to organizations that use AI/ML technologies to generate outputs (not just credit scores) and that then provide those outputs to third parties for use in their subsequent decision-making. The CJEU’s new approach may potentially render both the generator of the output and the user of such output subject to the EU GDPR’s ADM restriction, particularly if the user has heavily relied upon the AI/ML-generated output to make a “decision.” For example, in the scenario considered by the CJEU in the Schufa case, financial institutions may wish to consider the extent to which a human-led decision is being made on whether to grant the relevant individual the financial product and to what extent that human decision considers factors beyond the credit score provided by the CRA.
• Following the Schufa decision, certain EU member states, notably Germany, are considering national data privacy legislation to place credit scoring–related data processing on a firmer legal footing from an EU GDPR perspective. The proposed German legislation is expected to be adopted by the German parliament in the next few weeks.
• Relatedly, it remains to be seen whether UK courts adopt a similar interpretation under the UK GDPR as the CJEU did in the Schufa case with respect to the EU GDPR. In any event, the UK Parliament is currently considering amendments to the UK GDPR that may significantly narrow the scope of the ADM restriction under the UK GDPR.