AI: Principle of transparency and information requirements on the logic of the algorithm

Back

Article 5 states that transparency is one of the principles that must be applied when processing personal data. In line with this principle, the GDPR establishes a series of disclosure obligations for the Data Controller, which find their practical implementation in the privacy policy tool.

The GDPR then sets out an "enhanced" duty in cases of automated data processing, among which Artificial Intelligence softwares must undoubtedly be included.

Articles 13 n.2 lett. f) and 14 n.2 lett. g) of the GDPR, in fact, establish that the Data Controller, in order to guarantee a correct and transparent processing, must provide the data subject, at the time of obtaining his/her personal data, with information relating to:

f) the existence of automated decision-making, including profiling, referred to in Article 22(1) and (4) and, at least in those cases, meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject.

Likewise, Article 15 of the GDPR establishes the right of access to the same information for the data subject. It is important to underline that the Data Controller is required to be proactive in informing the data subject and not act merely on request by the data subject.

What then is meant by the terms "automated decision-making processes" and "significant information on the logic used" by the algorithm?

Let's try to go into more detail.

Automated decision making

With regards to the notion of automated decision making, the ICO, following the definition given by the Article 29 Working Party in the 'Guidelines on automated decision making concerning natural persons and profiling for the purposes of Regulation 2016/679', provides the following notion:

Automated decision-making is the process of making a decision by automated means without any human involvement. These decisions can be based on factual data, as well as on digitally created profiles or inferred data.

Examples of this include:

an online decision to award a loan; and
an aptitude test used for recruitment which uses pre-programmed algorithms and criteria.

Automated decision making often - but not automatically - involves profiling the person concerned.

There is no automated decision-making process when there is human involvement in the process (often referred to as "human-in-the-loop"): these are - as an example - cases in which the AI system formulates a prediction which is then subjected to scrutiny by a human subject, who makes a decision on the basis of that information. It is therefore an exclusively active intervention.

Significant information on the logic used: the guide of the British Information Commissioner’s Office

Providing meaningful information about the logic used by the algorithm means justifying how a certain result (output) was obtained from a certain starting point (input). In simple terms, you could say that you want to go and explain how and why from A you got to B.

The purpose of this explanation is to ensure accountability in case the decision was made by an AI and not by a human being.

The British Data Protection Supervisor has over the last two years focused a lot on the processing of personal data (think, for example, of the Guidance on the AI auditing framework).

As regards in particular the obligation to 'explain' IA, the very recent guide published by the ICO as a result of a collaboration with the Alan Turing Institute is of the utmost interest.

The guide is divided into three parts:

the first part is directed to DPOs and, more generally, to those involved in compliance with the GDPR;
the second part is directed to AI development teams and provides practical guidance on how to provide effective explanations;
The third part is directed to the company's senior management and covers the roles, procedures and policies required at the organizational level.

We will summarize the key passages of the first section, which are of interest from a strictly legal point of view, referring the reader to the full document (in English) for a more in-depth analysis.

First of all, the importance of the context when it comes to explaining decisions involving IA is immediately made clear: the ICO six main types of explanations from which the Data Controller should choose (or combine) on the basis of

the sector in which the AI model is used;
the impact on the individual;
the type of data used;
the urgency of the decision;
the audience to which the explanation is presented.

The explanations, according to the ICO (p. 20 et seq. of the Guide), can first be divided into two macro-categories:

Process-based explanations. These demonstrate that all best practices in product design and use have been followed during the decision-making process;
Outcome-based explanations. It is a question of clarifying the result of a specific decision by providing information on the reasoning followed in a simple and comprehensible language.

Then the ICO lists six types of explanations, which can be developed as process-based or outcome-based, providing guidance and checklists on how to implement them:

Rationale explanation: the reasons that led to a decision, pronounced in an accessible context and in a non-technical way.

Some of the questions to be answered by the Controller are:

What input characteristics, parameters and correlations have played a significant role in the calculation of the model result and how?
How can statistical results be applied to the specific circumstances of the person subject to the decision?

Responsibility explanation: Who is involved in the development, management and implementation of an Artificial Intelligence system, and who to contact for a human review of a decision.

One of the questions to be answered is:

How was it possible to make the design and implementation processes traceable and verifiable throughout the project?

Data explanation: What data was used in a given decision and how.

Some of the questions to be answered are

What data did you use to train the model?
Where did the data come from?
How was the quality of the data you used guaranteed?

Fairness explanation: The steps taken through the design and implementation of an AI system to ensure that the decisions it supports are generally impartial and fair, and whether or not an individual has been treated fairly;
Safety and performance explanation: the steps taken in the design and implementation of an IA system to maximize the accuracy, reliability, safety and robustness of its decisions and behavior;
Impact explanation: steps taken through the design and implementation of an AI system to consider and monitor the impacts that the use of an AI system and its decisions have or may have on an individual, and on a wider society.

It will be up to the data controller to identify, based on the context in which they operate, what types of explanations are prioritized and to document the reasons for his choice.

The example given by the ICO: explaining a cancer diagnosis carried out by an AI

In the example provided by the ICO in the Guide (Annexe 1), a model is provided for healthcare facilities where a step-by-step process is developed to build and present an explanation of a cancer diagnosis carried out by artificial neural networks to those concerned (patients).

In this case, the use of a two-layer explanation is recommended:

Explanation by the treating physician directly to the patient supported by a privacy notice delivered on paper or by email. At this stage it should be clarified:

How the data were labelled and why certain images are classified as tumors, how the result obtained by the AI team is used (rationale explanation);
The impact of the decision made by the IA on the patient (impact explanation);
Who the patient can turn to to challenge the diagnosis made by the AI (responsibility explanation);
How the computer security of the software was guaranteed, as well as the validity of the result (safety and performance explanation).

Explanation provided at a later time after meeting with the doctor in paper form, via email or via an app. At that stage you will need to explain:

The data that was used to train the model (data explanation);
The efforts made to avoid bias and discrimination in diagnosis (fairness explanation).

This is a very useful example given the continuous growth of applications of AI systems in healthcare, as we have highlighted in this article.

In conclusion, it is important to underline that - although the regulatory framework highlighted refers exclusively to automated decisions - when personal data is processed through AI (even with a "meaningful human involvment") it will always be necessary to ensure compliance with the general principles of treatment ex art. 5 GDPR.

In other words, we believe that the above explanation obligations are applicable to all processing of personal data carried out through AI, not only fully automated processing, on the basis of the provisions of Article 5 of the GDPR. This deserves even more attention when the processing relates to particularly sensitive data such as health data.