This project was designed at the intersection of Artificial Intelligence (AI) and urban governance in the context of smart and AI-powered cities. The use of AI to enhance the delivery of public services presents a multitude of opportunities, but it also introduces complex challenges. In particular, we frame those related to public participation and privacy concerns, by identifying and exploring key risks of data-driven and automation-based systems for cities, such as overreliance on data, data abuse and misuse, amplifying social inequalities and the erosion of citizens’ autonomy in decisions and situations that entail human-machine interaction in urban environments. In this sense, the project aims to discuss tradeoffs with local communities looking at the opportunities for AI to improve public services and accessibility while considering emerging challenges. By engaging both policymakers and citizens, the project aims to foster AI literacy, redefine the importance of public participation and support the development of AI systems that align with the local values and priorities. The specific objectives are 1) proposing an urban governance framework centered on participatory processes that prioritize people’s rights and values; 2) piloting the tools designed to assess and address the collective challenges of AI for cities; 3) delivering lessons learned from a case study. Ultimately, this research aspires to shape a more sustainable, inclusive and participatory inclusion of AI-based systems in cities by contributing with innovative urban governance frameworks and recommendations, ensuring that AI’s benefits are shared equitably among all city residents whilst respecting their rights.