BIG re-registration selection model

CIBG

A healthcare provider applying for BIG re-registration does not always have to substantiate this application with documentary evidence. A random and targeted sample is carried out. For the targeted random sample, the BIG register deploys an algorithm.

Last change on 17th of June 2024, at 14:00 (CET) | Publication Standard 1.0

• Publication category: Impactful algorithms
• Impact assessment: DPIA
• Status: In use

General information

Theme

• Work
• Health and Healthcare

Begin date

03-2022

Contact information

[e-mail]

Link to publication website

https://www.bigregister.nl/privacy#anker-1-geautomatiseerde-besluitvorming-en-profilering

Responsible use

Goal and impact

The aim is to select in the targeted sample mainly those applications that have a high probability of not meeting the legal requirements. On the contrary, those applicants that are likely to comply should be relieved of as much administrative burden as possible. This attempts to meet both the desire for effective protection of patient interests and the interest of professionals in keeping the administrative and financial burden on them limited.

Considerations

Assigning a risk category by the selection model aims to reduce the administrative burden on healthcare providers and work more on the basis of trust rather than mistrust. Reducing the administrative burden on healthcare providers requires assigning them to risk categories. Reducing the administrative burden is difficult to achieve any other way.

Human intervention

The selection model determines whether an application is selected for having to provide (objectively verifiable) documentary evidence.

The use of the algorithm does not lead to fully automated decision-making. After all, the final substantive assessment of an application based on the evidence provided is done by an employee.

Risk management

1. Preventing direct discrimination - when the files are assessed by the algorithm, no sensitive file characteristics (such as gender, age or having a foreign nationality) are shared on which the algorithm could make a direct distinction. Because these are not included as file characteristics, the algorithm cannot select on them either. Moreover, these characteristics are also not included as relevant criteria in the decision trees. This is to prevent direct discrimination on these characteristics;
2. Preventing indirect discrimination - although the decision tree cannot select directly on sensitive file characteristics, it is theoretically possible that a certain - seemingly innocent - characteristic is included in the tree, which has the practical effect that certain groups of healthcare providers are more likely to receive a high risk score. This is prohibited when there is no justification for doing so. During the development of the algorithm, this was considered and it was decided to show 'bias graphs' on the dashboard for the generated decision trees and the current decision tree, showing whether, as a result of a decision tree, certain groups of healthcare providers are selected relatively more often than other groups of healthcare providers. This is made transparent for the following characteristics: age, gender and having or not having a foreign nationality. Based on these bias graphs, a tree with the lowest possible bias or a tree where the indirect bias is objectively justifiable can be chosen;
3. In addition, the algorithm should be prevented from becoming self-validating. Therefore, in addition to the purposive sample, a random sample also takes place. The subsequent decisions are used to generate new decision trees;
4. It is not clear to the staff assessing the supporting documents whether the relevant application fell into the random or targeted selection. This is to prevent them from being unknowingly influenced by this information;
5. Regarding explainability, the following is noted: the decision trees clearly show the different threshold values. These threshold values are clear, concrete and also explainable in application to a concrete case;
6. Finally, it is noted that a DPIA has been prepared; it has been submitted to the FG for advice and subsequently approved.

Legal basis

BIG Act and the Periodic Registration Act Decree.

The explanatory memorandum includes the following in this regard: 'Applications for periodic registration will not have to be substantiated with supporting documents as standard. The self-declarations will be checked for content based on a random or targeted sample.

Links to legal bases

• Wet BIG: https://wetten.overheid.nl/BWBR0006251
• Besluit periodieke registratie Wet BIG: https://wetten.overheid.nl/BWBR0024841

Impact assessment

Data Protection Impact Assessment (DPIA)

Operations

Data

• General information about the case/application
• Personal information (this information is needed to understand any indirect bias)
• Information about hours worked
• Information regarding the content of the work performed
• Measures applicable abroad

Technical design

The algorithm used in targeted selection is a type of "decision tree classifier". It can be seen as a structured method of indicating which applications are likely to be at higher risk of not meeting the legal requirements for re-registration. Each node in this tree represents a specific question about the application, for example the total number of hours worked in the past five years. Based on the answers to these questions, the tree guides the decision across several branches until finally a conclusion is reached: either "select" or "do not select". In training, the algorithm learns to ask the most informative questions at each step based on the available data in order to divide the applications into smaller and more homogeneous groups.

The algorithm offers a choice of 16 different trees. Four different levels in terms of tree depth can be selected (3, 4, 5 or 6) and then the minimum size of the group is determined (100, 200, 300 or 400). Depth refers to the maximum number of splitting or decision steps the tree can have between the root (the start point) and the leaves (the end point). The minimum size of a group refers to the minimum number of applications for BIG re-registration that must be present on an end leaf of the decision tree.

The algorithm calculates the probability that an application belongs to a certain category, allowing us to classify applications as "medium risk" or "high risk". Only "high risk" applications are selected for verification in the targeted sample. This choice was made to ease the administrative burden on healthcare providers as much as possible ('low or medium risk' applications can still fall into the random sample).

External provider

Aigency