Please note: The algorithm descriptions in English have been automatically translated. Errors may have been introduced in this process. For the original descriptions, go to the Dutch version of the Algorithm Register.
DiDo
- Publication category
- Other algorithms
- Impact assessment
- Field not filled in.
- Status
- In use
General information
Theme
Begin date
Contact information
Link to publication website
Responsible use
Goal and impact
DiDo (Data in Data out) is about fast and automated reading of tables from suppliers into the Operational Data Layer (ODL) of a Postgres database. The ODL aims to organise uniform and reliable storage of the data provided by data suppliers. The supplier provides one-off or more frequent tables in .csv format (separated by semicolons). Those tables are read into Postgres based on a configuration file. Documentation is also generated that can be included in the project's wiki.
Considerations
DiDo puts data into a database and checks it for errors. First, a description of the data is used to automatically generate tables in Postgres and descriptions for the wiki. Then the data can be delivered, periodically as needed. This unburdens the user as much as possible, and in this case is an efficiency gain for the staff of the Policy Information Department.
Human intervention
DiDo is manually configured for new incoming data streams, and output can be easily adjusted manually.
Risk management
Risk management is covered by MinBZK's standard Responsible Disclosure Statement, available at https://github.com/MinBZK/DiDo/security.
Elaboration on impact assessments
DiDo has (virtually) no impact on people (rights) and is run in complete isolation within the Postgres database (data warehouse Policy Information). Further processing of data controlled by DiDo are intended only for internal central government operations.
Operations
Data
DiDo in its current and future form controls the following data flows arriving at the Policy Information Department, all fully anonymised and only concerning civil servants:
- P-Direkt (personnel data)
- Shuttel (travel data)
- UWV (benefits data)
- ABP (pension data)
The Open-Source version of DiDo has no access to this data and is just a shell to load and control similar flows.
Technical design
DiDo facilitates the user in storing data in the Datawarehouse and creating documentation on that data. DiDo has two phases:
- Data and Documentation Definition. This is (in theory) a one-off event in which the data is defined according to a set pattern, stored in Postgres tables and filed away as Wiki documentation.
- Reading data into the database. Once the tables are defined, the data can be delivered. Deliveries are stored in the table, along with the data quality. The delivery is documented and can be stored in the Wiki.
There are also a number of utilities that simplify dealing with suppliers and deliveries in the database.
External provider
Link to code base
Similar algorithm descriptions
- To correctly determine eligibility of residents, we use the application of an algorithm as a source of information. For example, by pre-determining whether the required information fields are filled in the application.Last change on 12th of July 2024, at 9:55 (CET) | Publication Standard 1.0
- Publication category
- Impactful algorithms
- Impact assessment
- DPIA, ...
- Status
- In use
- The algorithm is used by about 50 municipalities and, based on read-in data and answers given by the applicant, determines whether the applicant is eligible for any of the benefits to be applied for.Last change on 4th of April 2024, at 8:47 (CET) | Publication Standard 1.0
- Publication category
- Impactful algorithms
- Impact assessment
- Field not filled in.
- Status
- In development
- The algorithm is used by about 50 municipalities and, based on read-in data and answers given by the applicant, determines whether the applicant is eligible for any of the benefits to be applied for.Last change on 15th of November 2024, at 8:20 (CET) | Publication Standard 1.0
- Publication category
- Impactful algorithms
- Impact assessment
- Field not filled in.
- Status
- In use
- The algorithm is used by about 50 municipalities and, based on read-in data and answers given by the applicant, determines whether the applicant is eligible for any of the benefits to be applied for.Last change on 6th of September 2024, at 10:51 (CET) | Publication Standard 1.0
- Publication category
- Impactful algorithms
- Impact assessment
- Field not filled in.
- Status
- In development
- The algorithm in the software recognises and anonymises personal data and other sensitive information in documents. Governments regularly publish information related to the drafting and implementation of their policies (e.g. based on the Woo). This tool is used to render sensitive data unrecognisable in the process.Last change on 9th of January 2025, at 9:23 (CET) | Publication Standard 1.0
- Publication category
- Other algorithms
- Impact assessment
- DPIA
- Status
- In use