Citizen participation methods for Artificial Intelligence

Citizen panel

A citizen panel is a group of people selected to deliberate on public issues that affect them, in order to shape decision-making. Democratic theorist Mark Brown describes them as “temporary advisory bodies that involve lay people in cooperative deliberation informed by expert advice”.  This participation approach seeks to address concerns about “the competence of participants and the quality of their contributions to decision-making" when it comes to technically complex political issues. According to Crosby et al, citizen panels were originally modeled after the jury system.

Citizen panels have become increasingly popular as a form of democratic participation in the EU.  For example, the European Commission brings together citizens from all member states to discuss upcoming European level proposals.  With the help of a facilitation team, citizen panels work together in small groups of around 12, as well as all together in plenaries, to make recommendations to the Commission on policies and initiatives.

At the municipality of Amsterdam, citizen panels have been used in different ways to shape decision- making around AI projects. 

 For example, the Participatieraad, a citizen panel focused on Work, Participation, and Income, provided valuable input on the design and use of the Slimme Check algorithm.  This algorithm, which aimed to help determine which social assistance applications require further investigation, was eventually discontinued.  As welfare recipients and representatives of interest groups , the members of the participatieraad were expert stakeholders in this particular policy area and were able to give input on a use of technology which would impact them.

In 2024, the municipality’s Computer Vision team has created a citizen panel for their Responsible Scanning and Recognition project.  This is part of a broader collaboration with knowledge institutions, Amsterdam residents, and a technology provider focused on designing an image recognition solution for the public space which is ethically responsible, privacy friendly, and as secure as possible.  The citizen panel is present during the entire research and development process, working alongside experts and civil servants. This panel has been able to develop a nuanced understanding of the technology and its use bring forward their wishes and needs in the design and decision-making phases.

Working with a citizen panel involves a substantial time commitment from both the participants and the organizers.  It also involves going in depth with one group, rather than engaging a broader public.  However, this group has the opportunity to develop a deeper understanding on complex issues, which is particularly valuable when it comes to decisionmaking around new technologies.  A citizen panel also allows for more continuity in participation over long-term projects, which is often the case when it comes to innovation,  facilitating follow-up and accountability.

Sources:

• Crosby et al, “Citizens Panels: A New Approach to Citizen Participation”
• Mark Brown, “Survey Article: Citizen Panels and the Concept of Representation”
• European Commission, "European Citizens’ Panels A new phase of citizen engagement"
• Slimme Check: working with a citizen panel for innovation in the social domain

Socratic Design

Socratic Design is a philosophy and method that is applied to social issues where innovation in thinking and doing is desirable. It takes the residents of the city and their experiences as a starting point for creating wisdom for and about the city. The basic principle of the Socratic Design Method is that we cannot solve a problem with the same thinking that caused it. A reflection on our narrative is therefore an essential condition for arriving at practices and actions that contribute to a good future for the city. Through Socratic Design dialogues or labs, participants analyze assumptions that have led to where we are now and which assumptions we can adopt to contribute to a city in which we want to live and be.

Since the end of 2023, the Urban AI working group has been developing a Vision on AI for the municipality of Amsterdam.  As described in the Digital City Agenda 2023-2026, this Vision is a collaboration between the municipality, knowledge institutions, professionals, and most importantly, citizens. The team has chosen Socratic Design as the method to generate wisdom from citizens about the use of AI in the city, and identify key values to incorporate into the vision. To date, ten Socratic Design dialogues have been held as part of the Vision on AI participation trajectory.

This methodology is useful for diving deep into a topic an addressing fundamental questions.  It also holds value as an exercise in deliberative democracy.  As such, it lends itself well to the creation of visions, policies, and agendas.  It is important to note that the method is time-consuming, both in terms of the dialogue itself and processing the output.  The method also has specific rules, so it is important to have a trained moderator.  At the same time, Socratic Design is better suited to broader questions as opposed to very specific questions about technology, as participants also have a lot of agency in shaping the topic of conversation.

Sources:

• What is Socratic Design?
• Using Socratic Design to build Amsterdam's Vision on AI with citizens

User experience (UX) co-design

According to the Interaction Design Foundation,

“Co-design is a collaborative approach where designers work together with non-designers to create solutions. Designers act as facilitators and guide the participants through the design process. Co-design aims to harness the collective wisdom and insights of everyone involved, especially the end-users, to innovate and solve problems effectively.”

This approach enables the development of products or solutions with end users as opposed to for them, incorporating diverse perspectives and leading to more inclusive outcomes. It is also rooted in the context where it will be applied.   Although co-design can take many forms, such as workshops, studies, or brainstorming sessions, the principles of UX design require user participation throughout the entire design process, iteration and continuous evaluation.

User experience designers at the Municipality of Amsterdam are using co-design to work together with citizens with mobility impairments to develop a prototype of an AI-powered accessible route plannerwhich adapts to the accessibility needs of each user. This initiative is part of the PEER project, a “European collaboration investigating human-AI interaction with the aim of putting the user first, increasing transparency, and making the AI solution personalizable”. It is also part of the broader Amsterdam for All initiative, through which the municipality aims to make the city accessible for all citizens. The project aims to put citizens at the center of every step of the AI lifecycle, starting from design and development, all the way through evaluation. This is facilitated by collaboration with organizations such as Cliëntenbelang, an organization which aims to improve quality of life for people with disabilities.

This approach ensures that technological solutions are designed in a way that is usable and meets the needs of its end users, who are the most important stakeholders.  It gives citizens agency in the early phases of the AI lifecycle to ensure that their priorities and values are reflected in the technology.  This can have a positive impact on user satisfaction, trust, and ownership.  To achieve this, sharing findings and progress with participants is crucial especially within a lengthy and unpredictable innovation process.

 Sources:

• Interaction Design Foundation, "What Is Codesign?"
• Nesbitt et al, User Experience (UX) Design as a co-design methodology
• Working with citizens to design an accessible route planner

Survey

A survey is a common method to gather opinions and ideas from citizens on a given topic.  It can vary widely in length and structure, and can be administered digitally or in-person.  Surveys often also include some sort of contextual information about the topic.

At the municipality of Amsterdam, the Research and Statistics (O&S) department has a panel of citizens who have signed up to take surveys from the municipality.  As such, they are already interested in participating and typically have a high response rate.  

As part of a collaborative research project with Digital Rights House, the Research and Statistics department of Gemeente Amsterdam administered an online survey on May 2023 titled “Citizen participation in the use of algorithms by the municipality of Amsterdam”.  The survey focused on if and how citizens would like to learn more and participate on the topic of algorithms in the municipality. The thought behind conducting a survey was to use online channels to reach a wider audience than the other participation methods, through a format which takes relatively little time.  This method included an explanation of algorithms and examples of those used by the municipality, before diving into deeper questions about the if/ when/ how of citizen’s desire to participate on the topic of municipal use of algorithms.

In August 2023, the Computer Vision team sent out an online survey to the O&S panel to investigate the opinion of Amsterdammers about the use of camera images from scan cars.  The team is working on a technological solution that can recognize specific objects in public space. The findings from the survey not only guided the teams development efforts, but also contributed to the development of the citizen panel for responsible scanning.   

The benefits of this method are its scalability, especially if it is done digitally, and its ability to provide a high level overview using quantitative data.  At the same time, open-ended questions can provide additional depth and personal experiences.  Surveys can also be designed to be short, reducing the amount of effort required by participants.  There is always a balance to be made between breadth depth, and the influence citizens can have through this method is often limited. As such, surveys are often combined with other methods of participation

Sources:

• Survey: “Citizen participation in the use of algorithms by the municipality of Amsterdam”

Interviews

Interviews are a way to gather deeper individual perspectives and experiences about an important issue or initiative.  They are a very flexible and customizable approach to research as well as participation, but must be pursued with clear goals and questions in mind.  They can also easily be combined with other forms of participation in order to add additional depth and personal experience.   Stakeholder interviews usually intentionally target specific groups and individuals impacted by an issue, while outreach interviews entail going out into the field and speaking with citizens in a more spontaneous manner.  The core challenge of this approach is that it is not easily scalable.  This can be mitigated through working with partner organizations who have a presence “on the ground”.

As part of a collaborative research project with Gemeente Amsterdam, Digital Rights House went to parks and community centers in May 2023 to interview a handful of Amsterdam residents about the municipal use of algorithms and citizen participation. The defining characteristic of their approach, which they called an “outreach interview” is that citizens are met where they are, whether it be a park, community center, or other public space.  The “interview” takes the form of an informal conversation led by guiding questions, varying in length depending on the interest of the participant.

The guiding principle behind this approach is meeting the citizens where they are, instead of asking them to go out of their way to participate. As such, it allows for access to a different, and potentially more diverse audience than the other channels. It also prioritizes utilizing existing neighborhood spaces, from parks to community centers, in order to build a trusted, localized network, and as a result, better reach citizens. This participation approach can be valuable in reaching people who have less knowledge about new technologies, and who wouldn’t otherwise think to participate. 

Sources:

Outreach interviews on the use of algorithms by the municipality of Amsterdam

Citizen conversations

In the context of the collaborative research conducted by Digital Rights House and Gemeente Amsterdam, citizen conversations are small group discussions intended to facilitate dialogue and knowledge-sharing with and between citizens on a chosen topic.    

In April of 2023, a citizen conversation was held at Digital Rights House on the topic of citizen participation and the municipality’s use of algorithms. The goal was to allow for a dialogue with citizens where information about algorithms could be communicated in an accessible way, and where they could share their thoughts and ideas in a low-pressure setting. 

The citizen conversation aimed to address broader questions such as the perception of algorithms in general, and the perception of algorithms used by the municipality.  The algorithm register was incorporated into the conversation  to show examples of algorithms in use by the municipality.  This then prompted more discussion about the citizens’ desire to be informed about and participate in the municipality’s use of algorithms.

A 90-minute-long agenda was created with ice-breakers, an informative presentation on algorithms using two examples from Amsterdam’s algorithm register, guided group discussion, and collaborative sense-making.

Overall, the Citizen Conversation format is conducive to fostering more in-depth dialogue as well as information sharing.  This participation format allows citizens to learn from each other and reflect on their experiences.  In this case, the session was hosted by Digital Rights House; working with partner organizations in the future could make this approach more scalable, but only if there is a clear way for citizen input to reach relevant stakeholders in the municipality. 

Sources:

• Citizen conversation on the use of algorithms by the municipality of Amsterdam

As part of a collaborative research project with Digital Rights House, the Research and Statistics department of Gemeente Amsterdam administered an online survey on May 2023 titled “Citizen participation in the use of algorithms by the municipality of Amsterdam”.  The survey focused on if and how citizens would like to learn more and participate on the topic of algorithms in the municipality.

Goals

The thought behind conducting a survey was to use online channels to reach a wider audience than the other participation methods, through a format which takes relatively little time.  This method included an explanation of algorithms and examples of those used by the municipality, before diving into deeper questions about the if/ when/ how of citizen’s desire to participate on the topic of municipal use of algorithms. 

Description

A 15-question survey was developed specifically on the topic of citizen participation and municipal use of algorithms.  It was distributed online for a period of two weeks to 200 randomly selected participants from an existing panel managed by the Office of Research and Statistics.  Of the 200, 99 participants completed the survey.

The survey is divided into 5 sections.  The first section aims to evaluate the participant’s awareness of algorithms and their use within the municipality.  The second section provides definitions of algorithms, alongside three examples of algorithms used by the municipality. The examples are purposely chosen to represent the diverse use-cases for algorithms in municipal governance.  This is followed by questions gauging the participants' sentiments about the use of algorithmic decision making by the municipality, as well as their desire and ability to participate on this topic as citizens.  The third section dives deeper into how citizens would like to provide input on the municipality’s use of algorithms, both in terms of participation channel and in terms of phase in the algorithm lifecycle. The fourth section introduces the idea of the algorithm register, followed by questions to gauge the participant’s knowledge of and interest in using the tool.  Finally, the last section is meant to give participants an opportunity to provide feedback on the survey as a whole, and to leave any additional comments.

Building the survey

Developing the survey was an iterative process, with multiple rounds of feedback from both Digital Rights House and the Research and Statistics department of the Municipality of Amsterdam.  The following were the key themes of the feedback.

Firstly, the purpose of the survey should be introduced from the start.  In terms of structure, it should be relatively short and digestible, and participants should have the opportunity to opt out of a question or to elaborate on a response. Additionally, it is crucial to provide definitions and examples. Because we assume that most citizens have limited knowledge of algorithms and their use by the municipality, definitions and examples help level the playing field and provide a good starting-off point to anchor their answers in something real and specific.  These explanations should not use technical terms that may not be understood by all participants. Similarly, when asking about participation in different phases of the algorithm lifecycle, citizens might not understand what these phases are and what they entail.  It is important to explain, for example, what happens in the algorithm design phase and how citizens could participate.

Findings

The start of the survey focuses on citizen awareness about algorithms and their use by the municipality.

 As such, 82% of participants say that they know what an algorithm is, while only 52% say that they were previously aware that the municipality uses algorithms.  This shows a significant gap in awareness that presents a fundamental obstacle to citizen participation on the topic.  It also aligns with comments from other participation efforts which indicate that citizens have a very limited perception of what algorithms can be used for, but also of the municipality’s functions more broadly.

Participants were then asked if they care if a decision by the municipality is made using an algorithm.  More than half (57%) said yes, which supports the idea that citizens should be informed when an algorithm is in use, especially when it is used to make a decision that affects them.

Respondents were then given the opportunity to elaborate on their responses.  Their comments have been grouped below.

Concerns about the use of algorithms

Respondents had several recurring concerns about the potential harms of using algorithms.  Prominent among them was concern about prejudice and bias, specifically in the “characteristics, parameters, and predetermined goals embedded into the algorithms”.  This bias is linked to profiling people or dividing them into groups, leading to disparate treatment and unjustified intervention, as seen in the childcare benefits scandal (toeslagenaffaire).  The scandal is referenced multiple times as a reason for decreased confidence in algorithms and in the government. 

In general, respondents questioned whether the use of algorithms in public services is ethical and desirable.  This relates to the perception that algorithms are limited and lack “empathy and human thinking”, along with nuance and context. Blind trust in computers and algorithms, with no regard for errors or exceptions, is seen as dangerous; “when no appeal or human intervention is possible, it becomes scary,” wrote one respondent.  There is a common fear of generalization when “every situation is different”, and “citizens are people not protocols”.  Some even linked the use of algorithms to oversimplified policy.  There is also discomfort at the idea of humans being replaced by systems: “the world is cold enough without this approach”.

Beyond linking the use of algorithms to a lack of empathy, some respondents felt that algorithms are a way for the government to control citizens “from above”.  “Algorithms can lead to a tyrannical dictatorship”, one respondent argued.  This sentiment was also linked to concerns about privacy and the use of data.     As such, respondents argued that there must be checks in place and legal limits to the use of algorithms.  Furthermore, algorithms and the assumptions embedded into them should be open for public scrutiny; “I want to know how I and my fellow Amsterdammers are observed and analyzed.”

Acceptance of algorithms

However, not all respondents shared the same perspective on the use of algorithms.  Many expressed an accepting attitude towards the use of algorithms, although it was typically nuanced or conditional.  They saw algorithms as convenient and even necessary in some cases.  One participant wrote that “using this technology makes many processes much more efficient. I like the idea of knowing that the government and municipalities keep up with the times.” Another described algorithms as “specific and precise”, and a way to overcome administrative bottlenecks. Yet another emphasized that algorithms follow the same assessment and decision-making steps as humans.  Some were more measured with their responses, expressing that algorithms can be beneficial in most situations, but not all.  Another consideration was whether the algorithm was sufficiently checked, and if and how citizens could appeal decisions.  “I am not against algorithms”, one wrote, “but I am against them if they become an extra bureaucratic hurdle for citizens.”

Transparency and awareness

“The use of algorithms is unavoidable and also of social importance, but the use must be transparent and adequately documented,” wrote one respondent, highlighting the importance of transparency efforts by the municipality.  They continued to say that some level of accountability exists when a human makes a decision, in that they can be asked to explain their reasoning.  However, with algorithms, decision-making becomes a black box.  To that point, several respondents asked questions about how algorithms work and how they may impact a decision-making process, highlighting the need for awareness and education about algorithms in order to achieve transparency.

The next two questions were about desire and perceived ability to give input on the use of algorithms by the municipality.

68% of respondents were at least somewhat interested in giving input on the use of algorithms by the municipality, with 21% being very interested; only 18% said that they were not interested at all.  However, 40% of respondents said that they did not at all feel able to give their input on the use of algorithms by the municipality. This highlights the importance of removing barriers to participation and creating awareness of opportunities to provide input.

Respondents were given the opportunity to elaborate on their answers.  Many of them were interested in participating but either didn’t know how to participate or didn’t feel they knew enough about the topic.  Others were pessimistic about the lack of impact- they essentially felt that their contribution would be useless.  The comments have been grouped into a few common statements below.

I have never looked for opportunities to give input, but I would like to.

These respondents are interested or have an opinion on the topic, but don’t know how to participate.  One wrote the following: “I want to remain involved in decision-making about the neighborhood where I live, and about Amsterdam”.  Another said that “input from residents about its use is, I think, desirable; not about you and without you, but with you!”

I would not be able to make an impact.  Nothing can change now.

A couple of respondents felt that citizens have no influence on the use of algorithms by the municipality.  In particular, they felt that once an algorithm has been adopted, nothing can be done.  This is because they believe that algorithms are built on behalf of the municipality, which makes them beyond the control of citizens.

I am not sure what participation entails.

“I have no idea what kind of input the municipality expects,” said one of the respondents.  Others wondered how their contributions would be used, and even if their information would be incorporated into an algorithm.

I know nothing about algorithms and/or about what the municipality is doing in this area.

 Respondents feel that algorithms are difficult to understand, and some even stated that they don’t want to make the effort to understand them. Because they are not experts, they feel that they may not be able to give well-substantiated input. They also wouldn’t know where to start or who to go to; one asked whether there is someone “handling” algorithms. Transparency is also a key concern- “I have no insight into the use of this in the various departments of a municipality, so I cannot give an opinion on this,” one respondent wrote. 

            The municipality has not asked me to participate.

One respondent asserted that no one is currently commenting or participating on the topic of algorithms.  “So far, the municipality has not asked citizens anything, perhaps until now,” another added.

I am not interested, or I am not sure how this is relevant to me.

Some respondents simply did not want to participate on this topic or were not sure why they should.  Others specified that the responsibility of dealing with issues of algorithmic governance should not fall upon citizens: “this is a task of the municipal council within legal frameworks… I would not want individual citizens to participate in decisions about whether or not to use algorithms.”

The following series of questions aims to understand in what phase(s) of the algorithm lifecycle citizens are interested in participating, and illustrating to respondents the different forms that participation could take.

 Overall, it appears that citizens feel the most strongly about participation in the later phases of the algorithm lifecycle.  Specifically, 90% said that they would like to be able to contest a decision made using an algorithm (only 3% said no).  64% said they would like to have a say in whether the municipality decides to use an algorithm.

Still, a significant number of respondents expressed interest in shaping the city’s priorities for developing new algorithms and providing input into how algorithms are designed (47% and 43% respectively). What is interesting about these questions is the number of respondents who said don’t know/ no answer.  This likely indicates a lack of understanding of what happens in these early phases of the algorithm lifecycle, and what citizen participation could entail. 

Respondents identified the lack of knowledge about algorithms and their use as their primary obstacle to participation.  This highlights the importance of the register and other transparency initiatives in increasing awareness, and as a result, enabling participation.

Those who answered “other” were given the chance to explain what they perceive as obstacles to participation.  Here, several mentioned the role of the municipal council.  One mentioned that they preferred not to participate personally, but would rather leave the decision to experts and elected representatives. In contrast, another expressed that they had “wasted a lot of time” trying to give input through the council, but that nothing was ever done, and they were often “not even listened to”. Similarly, one respondent added that there is no space for respondents to share their opinion to the municipality.

In terms of how they would like to participate, most respondents preferred online channels, especially online surveys or polls.  However, this result is probably biased by the fact that the respondents are already used to taking online surveys.  Of the in-person options, small group discussions were the most popular, with 32% of respondents expressing interest. In the “other” box, participants were interested in participating through elected council members and appointed officials such as the ombudsman, or through more permanent bodies such as a committee or council.  They also expressed interest in educational events such as conferences.

The focus of the survey then shifts to the algorithm register.

 94% of respondents had never heard of the algorithm register. When asked if they would use the tool, 39% said yes, while 45% said they were not sure.  Respondents were then asked to clarify.  Overall, many still didn’t understand what the register is and what it contains.  However, some highlighted its potential importance as a source of information to enable them to better participate.

Respondents expressed confusion about the existence of the register and what it contains: “I now have no insight into what this register presents to me”.  Despite the short explanation provided in the survey, they still found it difficult to understand.  Others expressed that they could use it, but they didn’t know when or for what. One specified that they would find the register useful if it explains “the algorithm does and how it arrives at its advice and how it is checked”. 

While some respondents directly said that they would not use the register because it is too complicated and lacks a concrete impact, others were more interested.  They said that it seems useful as a source of information and would allow them to better participate and “exert influence”.  “I do want to know which algorithms are used in the system and also the motivation behind it,” emphasized one respondent.

A few responses also reiterated the need for citizens to know whether a decision has been made using an algorithm, so that they may then participate. “If I would like to challenge a decision, I first want to know whether the decision was made on the basis of an algorithm,” said one respondent, while another stated that the register would only be useful to them if they already knew that a decision was made using an algorithm.

Because this survey is exploratory and meant to shape future research, respondents were invited to give feedback on the survey itself. 

The most common feedback was on the clarity of the topic and making things easier to understand/ providing more examples.  At the same time, 74% rated the ease of understanding questions as a 4 or 5 out of 5.

Similarly, 75% answered 4 or 5 when asked if they are interested in the topic of this survey.  This is encouraging with regard to citizen interest in participating on this topic.  It must be taken into account that this is already a self-selecting group who already responded to the whole survey. 

Respondents were also given a final opportunity for feedback, from which the following themes emerged:

The survey could be more clear.

Some respondents commented that the explanations provided in the survey were not understandable for everyone, specifically regarding the purpose and deployment of algorithms. “I saw opportunities to explain it more clearly,” said one, while another suggested providing more examples.

“I would have found figures on this subject useful; for example, how many objections have been declared well-founded after cases that have been established using algorithms (think of parking fines, etc.)”

Overall, the topic was found by many to be “difficult and laborious”, and remains vague despite the information provided on the survey.

            The framing of the survey is biased

 “ I think this is a weird survey that seems very biased to me. Working with algorithms is made suspicious, while you wouldn't ask me to assess an employee's working method.” This respondent felt that the survey was too critical of algorithms, and implied that algorithms should be treated in the same way as human decision-making.

The survey is a step in the right direction.

“Understanding what can be done with algorithms and AI and how it works is of great importance,” shared one respondent.  They felt that efforts such as this survey could help to prevent misinformation about the use of algorithms.

            A follow-up to the survey is required.

A few respondents had follow-up questions after the end of the survey, expressing an interest in learning more about the municipality’s use of algorithms.  One highlighted the need for “feedback and follow-up plans”.

Conclusion

To begin with, there seems to be a gap in awareness among respondents about the municipality’s use of algorithms, which presents a fundamental obstacle to citizen participation on the issue.  Furthermore, citizens do care if a decision by the municipality is made using an algorithm and would want to be informed when the decision is made.  This is due to concerns about the use of algorithms leading to prejudice, bias, and lack of empathy.  Lack of visibility into the municipality’s use of algorithms is also a source of concern.

Furthermore, most participants are at least somewhat interested in giving input on the municipality’s use of algorithms, but feel that they are not currently able to.  Although there is significant interest in participating at every step of the algorithm lifecycle, respondents feel the most strongly about being able decide whether or not the municipality uses an algorithm, and being able to contest a decision made by an algorithm.  Lack of knowledge on algorithms and their use is seen as the primary obstacle for citizen participation.  This underlines the importance of efforts to increase citizen awareness on the topic, and to ensure that transparency tools such as the register are more widely accessible.  In fact, almost none of the respondents were aware of the existence of the register, and many were unsure of how it would be useful to them.  This presents an opportunity to increase awareness of the register once the new version is published, and to show citizens how this tool can best serve their needs.

Respondents prefer to participate through online channels such as surveys, but this result is not necessarily generalizable to the wider public. Most of them also expressed that the topic of this particular survey was of interest, and some even mentioned that it was the first time they have been able to give their input on the use of algorithms by the municipality.

As part of a collaborative research project with Gemeente Amsterdam, Digital Rights House went to parks and community centers in May 2023 to interview a handful of Amsterdam residents about the municipal use of algorithms and citizen participation. The defining characteristic of an “outreach interview” is that citizens are met where they are, whether it be a park, community center, or other public space.  The “interview” takes the form of an informal conversation led by guiding questions, varying in length depending on the interest of the participant.

Goals

The guiding principle behind this approach is meeting the citizens where they are, instead of asking them to go out of their way to participate. As such, it allows for access to a different, and potentially more diverse audience than the other channels. It also prioritizes utilizing existing neighborhood spaces, from parks to community centers, in order to build a trusted, localized network, and as a result, better reach citizens. This presents an opportunity to work with partner organizations, such as Digital Rights House, who has a focus on citizen listening and is well positioned to quickly roll out this type of participation/research approach.

Description

Groups of two or three representatives from Digital Rights House were sent to a local park and a community center with the goal of speaking to citizens about their views on algorithms, and whether or not they are interested in in participating on the topic. Three of the conversations were held with groups of two citizens, for a total of seven participants over four conversations.   Potential participants were approached and asked if they would be willing to speak to us as part of a research project on digital human rights.  The following served as the guiding questions for this research method, but the conversations were adapted to the context and participants.

1.   Have you ever heard about the use of algorithms by the municipality of Amsterdam? What comes to mind? 
2.   Do you care if an algorithm is used to make a decision that would affect you?  (Be prepared to give examples listed below if people need clarification)
3.   Would you want to give your input on the municipality’s use of algorithms? Do you feel that you’re able to? (If no and there is time, ask why not )

Typically, a broader question, such as “what comes to mind when you think of algorithms?” was used as an ice breaker to ease into the conversation. 

Findings

Interview 1

In general, this participant seemed to have a good understanding of algorithms.  Their attitude towards algorithms was nuanced, depending on the purpose of the algorithm, who is using it, and what data is collected.   They expressed some distrust of the government, and the idea that they would try to influence citizens using algorithms.  Another key concern was the risk of surveillance.  Nevertheless, they needed to hear examples of algorithms used by the municipality before giving their opinion. 

They viewed the use of algorithms to identify and help people who are at risk of falling into debt, for example, as positive.  Although it was not personally relevant to them, they saw this as an application that helps citizens, especially because people who are struggling financially or are marginalized are less likely to ask for help from the government.

To them, the party responsible for collecting and processing data was a key factor in determining their attitude towards the algorithm.  For the example of the parking control algorithm, they would prefer if an external firm owned the data, rather than the government. The motive behind collecting the data was also an important factor.

Although they thought the register was a good thing, they didn’t think that they would try to use it.  They felt that they were too old to keep up with new technological changes.  They also felt that participation should be the choice of every citizen.  They should have opportunities to participate, and they should be informed of these opportunities, but it should be up to each person. 

Interview 2

In contrast, the two participants in the second interview did not know much about algorithms.  They expressed a general uneasiness around technological change, as well as a distrust of government and banks. They felt stuck in a state of frustration and lack of understanding with everything becoming digital and automated. They were afraid of being controlled by algorithms and did not understand that they could be used to make services more efficient.  While they were worried about privacy broadly speaking, they also felt that they had nothing to hide from the government, and don’t take any precautions with regard to their data. 

Once they were given examples of algorithms from the register, they were more concerned about the outcome of an algorithmic decision – such as receiving a parking fine- than about the use of an algorithm itself.  They were not particularly interested in the algorithm register because they felt that they would not use it.

Interview 3

In general, the two participants in this conversation were very skeptical about the use of algorithms, although the topic remained a bit mystified by them. However, they were more worried about the commercial use of algorithms, for example in advertising, than about their municipal use.  This was partially because they did not expect the municipality to use algorithms.

After hearing about the examples from the register, they maintained a negative view on algorithms, and preferred that the same decision/ task be done by humans.  They felt that decisions on issues such as parking are quicker and harsher when they are automated.  Regarding welfare-related algorithms such as Vroeg Eropaf, they would rather the government stay out of their business instead of trying to intervene.

They were primarily concerned about privacy and what would be done with their information, such as parking data, for example. It was important for them to know if their data is handled by the municipality or by a private company.  Regarding the use of crowd sensing algorithms, they were concerned that cameras would be used for surveillance and facial recognition even if that wasn’t their stated purpose. 

In general, they liked the idea of the register and said that they would use it, because they do not currently feel that they know what the government is up to.  They were also curious if the register was populated by the government or by an external party, and as such whether it would be accurate.  They felt that they would want to know up front whether a decision affecting them is made using an algorithm.

These two participants immediately expressed that they had an opinion to share on the topics of digital rights and algorithms.  Although they would be interested to participate on this topic, they felt that they don’t know enough. Furthermore, they are open to different channels of participation, from a survey to a face-to-face conversation.

Interview 4

The only example of an algorithm used by the government that these participants knew about was the welfare fraud detection algorithm that was at the center of the benefits scandal. Due to the fact that their interactions with the municipality were essentially limited to administrative paperwork, they were surprised that they would have a need for algorithms. Their primary worries regarding the use of algorithms concerned the collection of sensitive data by the government, along with privacy and the impact of technology on children.

When discussing examples of algorithms from the register, their reactions varied significantly depending on the purpose of the algorithm and the data used.  Concerning the parking control algorithm, one mentioned that their spouse had gotten multiple tickets in a row from a scan car.  Nevertheless, they found that using an algorithm for this purpose is honest and straightforward, because the rules for parking are standard and should be followed in any case.

However, in the case of Vroeg Eropaf and other algorithms that deal with sensitive information and outcomes, this participant expressed fear that discriminatory decisions would be made based on their financial situation, for example.  They felt that the use of algorithms to make social decisions could reinforce the same cycle of disadvantage, and were concerned about which data is being collected and how it is being used – what happens if you are identified by this algorithm?  They were afraid that they would have a “record” that would stay with them, and that decisions would be made about their life and their children’s lives without their say.

When asked about sharing their opinion with the municipality on the topic of algorithms, this participant responded that “nobody asked”.  Although they would be interested in sharing their thoughts and opinions, they did not know that their point of view could be valuable to the municipality. Furthermore, they felt that they do not currently have the opportunity to participate.

Conclusion

Many participants did not realize that the municipality uses algorithms, partially due to a very limited view of the municipality’s functions.  While few said that they would use the register, its existence was seen as a positive step towards transparency.  Participants did not feel that they currently have a say in the municipality’s use of algorithms, but they believe that everyone should have the opportunity to participate.  At the same time, most of them felt that they did not know enough about the topic or did not feel legitimate to share their opinion.       

Regarding their attitude towards algorithms, participants were concerned about the use of sensitive data and being profiled by the government.  However, some felt that the social benefit of using algorithms for purposes such as welfare allocation might outweigh the costs.  Overall, privacy and control were key concerns about both public sector and commercial use of algorithms.  In fact, the lines were often blurred about algorithm and data ownership, indicating a need for further transparency on the issue.

 This participation approach proved to be valuable in reaching people who have less knowledge about algorithms, and who wouldn’t otherwise think to participate.  It also ended up including more older people, as well as busy moms who shared their insights while watching their kids.  This shows the importance of meeting citizens where they are in order to achieve more equitable and inclusive participation.

Conversations were most insightful when they were shaped around people’s concerns.  Furthermore, providing examples of algorithms used by the municipality was crucial in establishing the baseline level of understanding necessary for the conversation.  As such, if this approach is to be scaled, the interviewers on the field would have to be trained to be able to provide examples, and even answer questions or refer people to resources.

Based on the conversations, it seems that this approach may be best implemented through partner organizations as opposed to directly by the municipality. Firstly, multiple participants expressed distrust of the government, and asked if the interviewers were representatives of the municipality. Secondly, working with partner organizations would allow access to different neighborhoods and local community networks, and could also be integrated into existing citizen listening efforts, as was the case with Digital Rights House.

In the context of the collaborative research conducted by Digital Rights House and Gemeente Amsterdam, citizen conversations are small group discussions intended to facilitate dialogue and knowledge-sharing with and between citizens on a chosen topic.  In April of 2023, a citizen conversation was held at Digital Rights House on the topic of citizen participation and the municipality’s use of algorithms.

Goals

Citizen Conversations are meant to allow for a dialogue with citizens where information about algorithms can be communicated in an accessible way, and where they can share their thoughts and ideas in a low-pressure setting.  Sessions consist of a relatively small group of citizens,  and are structured in a way that encourages them to participate freely but purposefully.  For this study, the idea was to test out a citizen conversation model that could be iterated upon and replicated by different stakeholders and on different topics. 

The Citizen Conversation aimed to address broader questions such as the perception of algorithms in general, and the perception of algorithms used by the municipality.  The algorithm register was incorporated into the conversation by using it to show examples of algorithms in use by the municipality.  This then prompted more discussion about the citizens’ desire to be informed about and participate in the municipality’s use of algorithms.

Description

 The first citizen conversation was held on April 12, 2023, and was framed as a “trial run” to help Digital Rights House and the Municipality of Amsterdam determine how to organize such sessions moving forward, as well as to gather initial data on how to engage citizens on the topic of algorithms.  A 90-minute-long agenda was created with ice-breakers, an informative presentation on algorithms using two examples from Amsterdam’s algorithm register, guided group discussion, and collaborative sense-making. 

Overview of session

The session started with the question  “what do you think of when you think of algorithms?” 

Most participants immediately thought of social media, which reflects the fact that it is one of the most commonly known uses of algorithms that affects the average person on a daily basis.  None of the responses specifically reflected the use of algorithms in public space or for public services.  Many answers focused on the technical nature of algorithms, such as “coding”, “computers”, and “data analysis”, with some highlighting their innovative qualities, i.e. “new technology”, “smart tech”, and “automated”.  At the same time, others focused on their complexity (“hard to explain what it is”, “difficult technology”). This exercise also brought to light participant concerns about the risks and impacts of algorithms, such as “influencing our thoughts”, “manipulation”, and “data breach”.  One participant brought up the citizen “right to know” as a key element of the governance of algorithms.  In the discussion that followed, participants self-reflected on how the first things that came to mind regarding algorithms were shaped by their own exposure and experience.

The moderator then provided the following definitions of algorithms and algorithmic systems.

• Algorithm: A process which generates an output from an input (UN Habitat)
• Algorithmic System: A system that uses one or more algorithms, usually as part of computer software, to produce outputs that can be used for making decisions.
  • Functional definition: a system that uses automated reasoning to aid or replace a decision-making process that would otherwise be performed by humans (Ada Lovelace Institute)

After establishing a common definition of algorithms, the moderator then shifted the focus to the local context by asking “what do you know about the use of algorithms in Amsterdam? What comes to mind?”

In general, participants did not know much about the use of algorithms by the municipality, and didn’t realize many public services were powered by algorithms.  However, the examples brought up by some participants sparked very engaged discussion and a number of follow up questions, showing a curiosity and interest to learn more about this topic.  The topic that brought up the most debate was the use of surveillance cameras in public space.  Participants wondered if these cameras used algorithms to identify and track people.  They felt that what the cameras film, how they collect data, and what they are for is not transparent.  Other examples that came up were the automated traffic light system, which was seen as a potential positive use of algorithms for increased efficiency, along with parking enforcement.  Participants shared their negative experiences with automation, arguing that it is not always as “smart” as you think, and is often incapable of detecting nuance or dealing with exceptions.  As such, they highlighted the value of speaking to a human from the municipality, as opposed to a robot or machine.

At this point, participants were introduced to Amsterdam’s Municipal Algorithm Register as “a tool that citizens can use to learn more about how the city uses algorithms”.  The register was then displayed on a large screen, and participants were quickly walked through two algorithms in use by the municipality: Automated Parking Control, and Top 400/600.   This was used as a jumping off point for the main discussion activity of the session.  All participants were given time to answer a set of questions, detailed below. Afterwards, participants split into groups of two.  Each group was responsible for interpreting and synthesizing the responses to one of the questions, then presenting their conclusions to the rest of the participants.  This approach was inspired by the concept of collaborative sense-making described by Seebohm & Smith[1].

1. If an algorithm is being used in your city to make decisions that could affect you or people you know
   1. How would you want to learn about it, and what would you want to know?

Participants emphasized the need to know the purpose and goal for the municipality’s use of an algorithm, as well as what data is collected and how it is used. One brought up the desire to be informed about their (digital) rights and how to protect themselves from the potential harms of using algorithms. Regarding how they would like to learn about the use of algorithms in their city, participants underlined the importance of honesty and transparency, and mostly favored a multi-channel approach - “a website alone is not enough”.  Ideas included campaigns on different channels, such as social media, television, flyers, print ads, newspapers, and in public space (billboards and trams).  They also suggested direct communication, such as a letter or text message. Although media, specifically social media, were seen as powerful communication tools by participants, some said it can also lack credibility, especially if news does not come directly from the municipality.

1. Do you care that the decision is being made using an algorithm?

Almost all of the answers to this question were nuanced and conditional, showing that participants are willing to accept the use of algorithms if certain criteria are met.  Some brought to light the limitations of algorithmic decision making, such as the lack of sensitivity and nuance, and called for human controls and checks. Others emphasized that the use case of the algorithm matters; while the automation of simple decisions for increased efficiency may be acceptable, dealing with more sensitive personal information and decision-making is less accepted.  In addition to human oversight, other key conditions included transparency and ethical use.

1.   If the municipality is thinking about using a new algorithm in Amsterdam
2. a)    What should they take into consideration?

The importance of centering citizens, and humans more broadly, was the salient theme in the responses for this question; “[the citizens] are their city so they and their wellbeing should be taken into account”. The group tasked with commenting on this question separated the answers into the risks and rewards that should be identified before an algorithm is rolled out, along with who may be affected.  “Risks can be misuse of privacy, violation of human rights, and long-term safety in public space.  Benefits like citizen well-being and the obvious efficiency/ accuracy should also be considered” they wrote.    

1. b)    Who should they talk to?

Along the same lines, participants felt that the municipality should initiate open discussion with citizens and affected communities when considering using a new algorithm in the city.  They suggested involving other stakeholders such as civil society, activists, and people representing citizens and working for human rights.  Furthermore, the municipality’s strategy could involve going to community centers and housing developments at the neighborhood level, as well as reaching out to people by mail.

1.   Do you want to give your input on the use of technology in and by the municipality? Do you feel that you are able to?

Although the responses varied, the participants summarized them well: “In general, everyone wants to have the right and ability to be included.  While not everyone feels like they can, they would like to have the option to engage and give their opinions.”  Even those who did not necessarily feel the need to participate wanted to have the option to do so.  Some specifically mentioned that everyone should be able to participate through channels such as the algorithm register or through surveys.  While one respondent stated that they do not currently feel able to participate, most felt that they could, although they were not clear as to how. 

The session was concluded by asking participants what meaningful citizen participation means to them.  One described it as a process “whereby a representing group of citizens is actively engaged, made aware/ educated and involved in the decision-making towards a particular outcome (and that their impact actually carries weight)”.  This definition involves three phases: the awareness-raising that is required before participation is possible, participation which takes the form of involvement in decision-making, and follow up on the participation.  These themes were recurrent in many of the answers, particularly the desire for citizens to have impact or influence.  One response also emphasized the need for complex topics such as algorithms, and how they are used, to be explained to citizens in an accessible way in order to enable meaningful participation.  Others described the nature of the participation more specifically; for example, conversation should be open, it should be at the neighborhood level, and it should occur both in person and online.  Furthermore, citizens should be ‘representative’, meaning that “the people involved in participation efforts represent/ reflect the people affected as well as possible.”  Finally, one respondent added that citizens should be able to directly speak and ask questions to someone from the municipality as part of the participation process.

Participant Feedback

Participants were also invited to provide feedback on the session’s content and structure.  To begin with, it is helpful to ask questions to evaluate people’s baseline level of understanding before diving into examples.  Defining key terms such as algorithms and algorithmic systems is also critical.  Participants also found a walk-through of the algorithm register helpful, albeit information-dense. The approach to presenting the register within the time constraints at hand needs to be further developed.  For this type of in-depth discussion, participants favor small groups from 6-8 participants.  Larger sessions should be broken up into smaller breakout sessions in order to better facilitate conversation.  Approaches where people are given time to write down their ideas (ie using post-its) then share them with the group can make it easier for people who are shy and hesitant to participate.  After the session, resources should be provided for participants who want to learn more about the topic.

They also provided the following input about participating on the topic of algorithms in general.    Citizens may be more likely to participate on this topic via surveys or other virtual channels than in person sessions such as this one.  They would want to participate in more involved, in person discussions on issues that immediately impact them, their family, and their immediate community or neighborhood (for example, the local school attended by a family member).  With technology and algorithms that are not necessarily visible and present, it’s harder to feel that there’s an immediate impact, and therefore it is harder to feel the need to participate in a more engaged way.  Citizens might be just as, if not more, concerned about the use of algorithms by the private sector than by the municipality, and may want to know the municipality’s response to those concerns.

Conclusion

When participants think about algorithms, public services do not immediately come to mind. As such, most did not realize that many public services in Amsterdam are powered by algorithms. However, once examples were brought to the table, participants were very interested and curious about the topic, opening up a space for questions and discussion.  Concerns that emerged related to the use of algorithms were the risks of surveillance and lack of human oversight. 

When an algorithm is being used to make a decision that could affect them, citizens want to know what data is being collected, whether there are privacy measures in place, and whether the use of the algorithm is properly justified.  Their acceptance of algorithmic decision-making in general is conditional upon factors such as human oversight and ethical use.  They value transparency and communication from multiple channels about when an algorithm is being used. 

If the municipality is considering implementing a new algorithm, participants feel that they should strongly consider risks and benefits to impacted citizens.  They should also initiate open dialogue with these citizens along with other interested stakeholders, such as civil society and advocacy groups. 

Participants believe that everyone should have the opportunity to give their input on the municipality’s use of algorithms, but not all felt that they are currently able to.  To the group, meaningful participation is accessible, allows for open dialogue, and has a clear impact and follow up.  It is also as inclusive as possible.

Overall, the Citizen Conversation was conducive to fostering more in-depth dialogue as well as information sharing.  This participation format allows citizens to learn from each other and reflect on their experiences.  However, it is quite involved and time-consuming, which means that it would probably only attract people who have the time and who are interested in the topic.  In this case, the session was hosted by Digital Rights House; working with partner organizations in the future could make this approach more scalable, but only if there is a clear way for citizen input to reach relevant stakeholders in the municipality.  Furthermore, these sessions could benefit from having a representative of the municipality available to answer questions and concerns about the use of algorithms.

[1] Seebohm, L., & Smith, N. Learning to Listen Again: How people experiencing complex challenges feel about engagement and participation through the Covid-19 pandemic. Centre for Public Impact UK.