AI-driven support for citizens making a signal (melding)

Value for Amsterdam

The City of Amsterdam addresses every citizen's report manually, even when no new follow-up action is required. In such cases, the City typically informs citizens about a planned solution or explains why an issue cannot be resolved due to policy constraints or jurisdictional limits. These cases, which constitute 25-30% of reports, are currently handled using the same processes as other reports, leading to delays, dissatisfaction, and an unnecessary allocation of municipal resources. 

The envisioned Agentic RAG solution offers an opportunity to enhance citizen satisfaction by automating responses for these cases. Citizens receive fast, empathetic, and context-specific feedback without waiting for several days. This improves transparency, ensures citizens understand which actions are being taken (or not taken) and why, and provides a user-friendly interaction tailored to the reporter’s needs. 

The solution also reduces the workload for municipal staff, allowing them to focus on complex reports requiring human intervention. By automating simple reports, more reports can be addressed within the City’s pledge of service (servicebelofte). Moreover, the system’s scalability enables adoption by other municipalities and extension to other communication activities, such as subsidy or permit requests, while complementing existing communication methods. 

User Research and Key Findings 

User research was central to the project’s development. Surveys and user tests showed that citizens are generally open to AI-generated responses for simple reports, provided a human backup option is available. Factors such as efficiency, simplicity, and transparent communication about AI usage were critical to user acceptance. Personalization features, such as multi-language support, further enhanced accessibility. However, trust in the system depends on addressing concerns about privacy, ensuring empathetic and accurate responses, and proving AI’s reliability and effectiveness in handling reports. 

Technical Implementation and Results 

The implemented agentic RAG architecture leverages modular tools to dynamically gather information and resolve reports. The system currently uses GPT-4o but is designed to accommodate alternative models, depending on language support, scalability, and task-specific needs. While the solution shows strong performance in retrieving relevant content from multiple municipal data sources, improvements are needed in communication style and tone, which future iterations aim to address. 

The system’s modularity ensures adaptability, allowing for iterative improvements such as integrating additional tools or refining prompts for specific goals. Despite promising results, challenges related to system complexity, scalability, and known LLM limitations, such as randomness and transparency, remain areas for further optimization. This approach provides a solid foundation for future enhancements to meet citizen and municipal needs effectively. 

Ethical Considerations

Ethical workshops evaluated the solution against the Amsterdam Vision on AI, which emphasizes reliability, transparency, and inclusivity. Opportunities included increased accessibility through multi-language support, more empathetic responses, and improved transparency by grounding AI answers in municipal policy. However, risks such as bias in language processing, unequal service quality, and privacy vulnerabilities were also identified. To mitigate these risks, the system must ensure clear communication about its AI-driven nature, avoid stereotyping in responses, and maintain alternative communication channels to serve diverse citizen needs. 

 Climate Impact and Sustainability 

The proposed Agentic RAG system has a notable environmental impact due to its reliance on resource-intensive LLMs, with factors such as model size, frequent LLM calls, and prompt length contributing significantly to energy consumption. Multi-purpose models, such as multimodal LLMs, further increase resource usage unnecessarily for specific tasks. 

To mitigate this impact, the system should prioritize task-specific models, reduce LLM calls, and optimize prompt and response lengths. Transparent tracking of resource usage, such as through self-hosted open-source models, will enable better comparisons with current processes and guide sustainable design choices. By balancing performance needs with environmental considerations, the system can align with Amsterdam’s commitment to climate responsibility while maintaining scalability and efficiency. 

Conclusions

The envisioned solution enhances citizen satisfaction by delivering fast, empathetic, and accurate responses to citizens reporting issues in public spaces. It improves response times and reduces manual workload for municipal staff, addressing citizen reports that do not require new actions. Up to 25-30% of reports could be handled automatically in this way. 

Citizen feedback demonstrated openness to AI-generated responses for simple tasks, provided key factors such as efficiency, simplicity, and transparency are maintained. Personalization features like multi-language support could enhance accessibility, though the solution must still prove its reliability to users. The technical results of the Agentic RAG framework show promise in retrieving and synthesizing relevant information, but improvements are required around tone of voice and bias mitigation. Additionally, the environmental impact of frequent LLM usage calls for alternative models and architectural refinements to reduce emissions and operational costs. 

Recommendations for Next Steps

The AI Lab recommends engaging the team responsible for the existing Signals system to discuss ownership and integration of the envisioned solution. Following the City of Amsterdam’s "Generative AI Development Process" will ensure alignment with municipal AI policies and ethical standards. 

Minimal performance and ethical benchmarks should be established, such as clear standards for content accuracy, tone of voice, and transparency. Research and iteration should focus on validating these requirements, starting with specific scenarios like duplicate or policy-related reports. Scalability must be considered to allow the gradual inclusion of other scenarios. 

User testing should begin in controlled environments with small groups, gradually scaling up while implementing improvements based on feedback. These steps will ensure the solution delivers its intended benefits to citizens and municipal staff, ultimately enhancing the overall reporting process. 

By addressing these recommendations, the City of Amsterdam can leverage Agentic RAG responsibly, aligning its implementation with the city’s commitment to trustful, human-centric, and sustainable AI innovation.