ChatAmsterdam

Municipality of Amsterdam

Study on deployment of a Generative AI chatbot for employees of the Municipality of Amsterdam

Last change on 21st of October 2024, at 15:57 (CET) | Publication Standard 1.0

• Publication category: High-Risk AI-system
• Impact assessment: DPIA, ...
• Status: In development

General information

Theme

Organisation and business operations

Begin date

11-2024

End date

03-2025

Contact information

[e-mail]

Link to source registration

Dit is een interne repository die alleen toegankelijk is voor medewerkers van de gemeente Amsterdam.

Responsible use

Goal and impact

Generative AI offers many opportunities to make work easier such as finding information, summarising documents, analysing data or helping to write papers.

But this technology also has downsides: training AI takes a lot of energy and natural resources, owners of the source material for training the models are not paid, and the model - depending on the sources - can produce racist or other ethically undesirable results.

Apart from these general criticisms, there are also risks to the organisation. Documents offered to a chatbot can be used by the supplier to further train the model. What if they contain sensitive information? Or if an official consults the bot to find information to support decision-making, while the model answers based on dated or outright incorrect information?

Research is needed to explore the value of Generative AI for Amsterdam while identifying practical risks. Amsterdam employees will therefore start using an internal Generative AI chatbot for a limited period of time: ChatAmsterdam. This chatbot is built with Open AI's GPT4.0 language model and is not otherwise modified or supplemented with (internal) training data.

During the research period, usage behaviour, value and risks will be assessed to determine whether we will use this technology in the future.

Considerations

We know that Generative AI is already being used by employees AND we recognise the opportunities and risks. There are plenty of applications to consider for the organisation: both internal and external. But before we allow Generative AI (within the compliancy requirements) within the Municipality of Amsterdam, research is needed. How do people communicate with a bot? What kind of (sensitive) information is sent along in the prompts? What are the results used for?How valuable is the technology for the Amsterdam municipality and the city's citizens and entrepreneurs?

These answers will help the Amsterdam municipality deploy the technology safely, responsibly and in a targeted manner.

Human intervention

The chatbot and underlying language model do not make independent decisions, nor are they part of an automatic decision-making process without human intervention.

All generated answers are viewed by the user who asked the question/command to the chatbot. The user then proceeds to work on the answer, for example by adopting it in part or in full or by asking further questions to the chatbot. The user is always between the generated answer from the chatbot and the task or process the answer is used for.

Risk management

ChatAmsterdam is offered as an internal chatbot to internal employees of the Amsterdam municipality.

The chatbot application, data and hosting environment comply with applicable BBN2 security standards to minimise risk of data leaks and security incidents.

All prompts entered are stored and manually reviewed for potential ethical risks, bias and possible data leaks.

In addition, there is a two-tier filter function. Microsoft's prompt filter filters all prompts for dangerous and unethical issues such as discrimination, suicide or abuse. The second filter is an internally developed prompt filter that specifically looks for risky content such as use of sensitive data, ethnic profiling et cetera

These risk prompts are stored separately and submitted to the relevant Risk/AI officers for content review.

Elaboration on impact assessments

Amsterdam is currently conducting the ethical impact test in the form of an Ethical Leaflet. This has been completed for this algorithm.

Impact assessment

• Data Protection Impact Assessment (DPIA)
• Ethische Bijsluiter

Operations

Data

ChatAmsterdam uses the Open AI LLM, specifically GPT4.0. This is the same language model (algorithm) used in ChatGPT.

This language model has not been trained further with additional (internal) data. The language model is used as is, with no changes or fine-tuning.

Technical design

ChatAmsterdam as a Generative AI chatbot uses the GPT4.0 language model in its current form without adding any (internal) training data or other data. It is the same language model that ChatGPT uses.

ChatAmsterdam works in a similar way to ChatGPT. Employees can ask a question or issue a command (called a prompt) to the chat application and then receive a generated response. An example of this is summarising or rewriting texts.

The language model processes the entered question or command and included information and then sends the generated answer back to the user.

External provider

The algorithm (in this case, the language model GPT4.0) is provided by Microsoft through the Azure cloud platform.