Game-based assessment (GBA) in selection of government-wide traineeships

Ministry of the Interior and Kingdom Relations

A GBA is one part of the selection process. Candidates play several games to give an impression of the fit with the trainee profile. The test scores are visible in the candidate file as a report.

Last change on 10th of September 2024, at 14:28 (CET) | Publication Standard 1.0

• Publication category: Impactful algorithms
• Impact assessment: IAMA, DPIA
• Status: In use

General information

Theme

Work

Begin date

09-2021

Contact information

[e-mail]

Link to publication website

https://www.werkenvoornederland.nl/starters/traineeships/het-rijkstraineeprogramma https://www.werkenvoornederland.nl/starters/traineeships/rijks-hbo-traineeprogramma https://www.werkenvoornederland.nl/starters/traineeships/traineeship-rijksinkoopprofessionals https://www.werkenvoornederland.nl/starters/traineeships/rijks-i-traineeship https://www.werkenvoornederland.nl/starters/traineeships/internationaal-beleidsmedewerker-buitenlandse-zaken

Responsible use

Goal and impact

The main purpose of deploying the GBA is to use the test results in conjunction with the results of other selection items to make an objective selection decision and ensure more equality in the mode of selection: no game experience is required for the GBA. Sub-goals of the deployment of this algorithm are:

• A match based on qualities and talents rather than education level.
• A structured approach that is the same for all candidates.
• Making the test more accessible to candidates and appropriate for the target group.
• Provide a way (in combination with other selection components) to select from many applications, which is not possible manually.

Considerations

These are the following public values that inform the use of the algorithm:

Public values to which the algorithm contributes:

• Equality (equal opportunities): This is where implementation research was done prior to the deployment of the GBA. The GBA targets a wide range of competences and talents which promotes diversity in the selection decision.
• Efficiency: There are very high numbers of applications for the State Trainee Programme. The government does not have the resources and capacity to handle this manually and is therefore forced to opt for a different selection mechanism that allows for more efficient use of limited government resources.
• Effectiveness: An effective government needs staff that is a good match for the tasks at hand. The GBA takes a broader look at competences and talents than previous tools (which mainly focused on intelligence). This allows a more sophisticated search for suitable candidates.

The public values that may be compromised by the use of the algorithm, or at least should be seen as a concern, are:

• Equality: It is essential that the algorithm does not contain a bias that puts certain groups at a disadvantage.
• Explainability: It is important that the GBA is and remains explainable and that there is a clear procedure for candidates to request more information on how a result was established. This procedure should also be actively communicated.
• Accessibility: Accessibility as a public value may be at stake: not everyone can make the GBA. Sufficient customisation should be possible. Candidates are given the opportunity to share circumstances that may affect the selection procedure with Leeuwendaal and/or the State. All comments are gone through and assessed on an individual basis. Sometimes exemptions are granted on that basis.

The values outlined here should be seen as points of consideration, including for possible design changes and future developments. It is important to note that the negative impact on values formulated here will be mitigated by taking measures.

Human intervention

Based on the scores coming from the GBA, a report is fed back to the applicant. These scores are part of the file, which along with other data such as CV and motivation questions are used in the selection process. No selection decisions are made without the intervention of a State employee.

So there is definitely "human in the loop" because the recommendation is made by a person and the score is only part of this decision.

Risk management

• The effectiveness of the GBA is reviewed annually to ensure that the algorithm is and remains effective.
• To prevent the GBA algorithm from being used for other purposes for which it is not intended, it has been agreed that a separate IAMA will be carried out for each use.
• Because the deployment of an algorithm can raise questions, the provision of information to those involved is reviewed annually, partly on the basis of the questions asked during the application periods of State trainees.

Legal basis

The Civil Service Act 2017 includes that "Public employers may conduct an investigation into the suitability and competence of an employee for a civil service position". The following applies to the GBA:

The legal basis lies in the fact that a company has the right to hire a new employee: a government employer has the right to conduct an investigation into the suitability and competence of an employee. However, there is nothing to say about the procedure. However, there is an application code for this. This application code tells us more about the use of digital assessments:

"If the organisation uses data recruitment in pre-selection, such as digital assessment, Artificial Intelligence (A.I.) or algorithms, for example, these should be validated and transparent. With the organisation using A.I and algorithms, the potential risks and shortcomings thereof should be clear. At all times, care and confidentiality should be ensured and no health-related questions should be asked. The applicant will receive a reasoned result after participation, if requested by the applicant".

The use of the DPIA and the IAMA ensures that the assessment is validated and transparent.

Links to legal bases

Ambtenarenwet 2017: https://wetten.overheid.nl/BWBR0001947/

Elaboration on impact assessments

An IAMA and DPIA were carried out as part of the privacy/ethics 'by desgin' process.

Impact assessment

• Impact Assessment Mensenrechten en Algoritmes (IAMA)
• Data Protection Impact Assessment (DPIA)

Operations

Data

Only data that can be extracted from the GBA completed by the candidate is used. The algorithm only uses data necessary to determine the final score (data minimisation). For example, no demographic information is made available to the models.

The exception here is when a candidate indicates having a neurodiverse condition. In that case, adjustments are made to allow the candidate to still take the test if the candidate is able to do so. Employer can also choose to exempt candidates from the GBA.

Technical design

This is a rule-based algorithm. Thus, there is no self-learning algorithm. It uses only the data obtained during the test. The answers entered by the candidates and chosen method during the test determine the scores on the competences. These competences are aligned with the State and determine the fit with the trainee profile. The test is started from the candidate's file through a technical link. No personal data are shared in this link. When the candidate is ready, the scores are calculated and retrieved from the GBA and the competency scores are displayed in the candidate's file.

External provider

Artic Shores: Developer of the algorithm Eelloo: distributor of the algorithm Leeuwendaal: contract partner