This research was conducted by Andrea Lombardo in collaboration with the AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Diederik Roijers
Supervisors: Tim Alpherts (UvA)
The global issue of urban accessibility, particularly sidewalk accessibility, significantly impacts
the quality of life for around 16% of the world’s population. Traditional data collection methods
for sidewalk assessment are laborious, costly, and time-consuming. Although supervised learning
offers cost-effective scalability, it requires large, high-quality and diverse training datasets,
which are scarce in the urban accessibility domain. To overcome these challenges, we explore
the potential of self-supervised techniques in a computer vision pipeline to localize obstacles on
sidewalks without relying on large-scale datasets. Our work presents a modular pipeline comprising
deep learning models for unsupervised object discovery and a semantic segmentation
module to differentiate urban context aspects. The inclusion of semantic information improves
the model’s performance across all evaluations. We introduce a novel qualitative evaluation
framework and a user-friendly Graphic User Interface (GUI) to facilitate human-in-the-loop
expert evaluations. Additionally, we demonstrate the pipeline’s robustness across diverse geographical
inputs. Our work contributes to AI-assisted urban accessibility by bridging the gap
between research and real-world applications.
This research was conducted by Andrea Lombardo in collaboration with the AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Diederik Roijers
Supervisors: Tim Alpherts (UvA)
This research addresses the challenges of extracting indoor venue accessibility information from noisy datasets, utilising the TinyBERT model with transfer learning on Google reviews datasets.
This research was conducted by Mylène Brown-Coleman in collaboration with AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Iva Gornishka.
Supervisors: Xander Wilcke & Iva Gornishka
Digital twins (DTs) are a common way for city planners and citizens to visualize impact of new policy decisions, simulate scenarios, and plan for disasters. The more detail these DTs have, the more useful they can be. Windows and doors, also known as building openings, are a critical detail missing in Amsterdam’s DT. While previous works use images, this paper investigates the usefulness of a lidar-based point cloud for building opening detection by proposing a pipeline to this end.
This paper creates a dataset for semantic segmentation and a dataset for instance segmentation for window and door classes. We begin with isolating individual buildings from larger scans using their footprint accessed from a government building database. Then, each building cloud is passed to a RandLA-Net neural network for point-wise semantic segmentation. After inference, the cloud passes through an internal false positive rejection (IFPR) module that makes use of the building footprint to rectify incorrectly labeled points in the building’s interior. Then, points are clustered into instances by an adaptive DBSCAN algorithm that derives information from data inherent to each point cloud. This pipeline shows promising results on a small-scale dataset, detecting window and door points with a mIoU of 37.81, and 40.57, respectively. Post-processing yields a 3D IoU score of 77.1 for windows and 81.32 for doors in the ideal inference case.
This paper tackles the main challenges of lidar-based building opening detection and discusses ways to mitigate these challenges in future work.
This research was conducted by Willem van der Vliet in collaboration with the AI Team and the 3D Amsterdam team of the City of Amsterdam.
Involved civil servants: Daan Bloembergen & Niek Ijzerman.
Supervisors: Daan Bloembergen & Holger Caesar.
A Multi-Objective Optimization problem focuses on producing optimal paths considering different objectives. This project aims at developing optimal paths for disabled citizens in the city of Amsterdam by capturing their preferences, thus integrating an Interactive User Approach. This project builds on previous research, which utilizes a non-parametric approach, Gaussian Process with Expected Improvement, and, instead, provides a parametric approach, Bayesian Logistic Regression with Thompson Sampling, to tackle the problem. This research incorporates Thompson Sampling as the sampling method for exploration-exploitation trade-off.
The experiments conducted in this paper demonstrate that Thompson Sampling is slow in performance given its stochastic nature, and the parametric approach performs at par with the nonparametric approach. Hence, the results of the experiments suggest that approaching the problem with the use of a parametric approach is a promising method due to its advantage of maintaining performance and offering reusability.
This research was conducted by Aditi Mishra from the Department of Advanced Computing Sciences - Faculty of Science and Engineering, Maastricht University in collaboration with the AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Diederik M. Roijers and Shayla Jansen
Supervisors: Diederik M. Roijers, Shayla Jansen
Cities should be easy for everyone to get to and move around in. But this doesn’t always happen. While cities are putting effort in creating a more open environment for people with disabilities, these efforts do not always suffice. This is not just due to a lack of resources, but also the way they are being used.
Right now, inspectors have to go out and physically check the sidewalks and streets to gather information about how accessible they are. While this is the best way to know about the condition of different parts of a city, it takes a great amount of time. But what if there is a way to avoid this slow and tedious process.
This is where Project Sidewalk comes in. With this project research is being done in using AI to do these inspections, instead of doing them manually. However, creating an architecture to do these inspections has some drawbacks. One major hurdle to overcome, is that training an Artificial Intelligence model requires a lot of data.
Project Sidewalk is a public crowdsourcing tool, made by the University of Washington, designed to overcome this hurdle. Using this tool everyone, whether they are an expert or just an interested individual, can start validating the sidewalk accessibility features of a certain city. By following the labeling guide, the user can virtually walk through these cities, and point out inadequate accessibility features for people with disabilities.
This research was conducted by Kasper Verhavert in collaboration with the AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Diederik M. Roijers
Suppose you want to go to the Rijksmuseum on foot. Navigation apps determine the shortest path from your location to the Rijksmuseum based on one objective (i.e., criterion), which is most often the length of the path. However, considering only the length of the path is often not sufficient for people with mobility issues and those who use assistive devices, such as wheelchairs, strollers, and canes. They have different and many needs with respect to the accessibility of sidewalks. For example, they might prefer to take a longer path which has fewer crossings than a shorter path with more crossings. They might even prefer to take only those sidewalks which have a certain width. Hence, this accessibility, and especially the lack thereof, of the paths results in additional and different objectives. This research is part of the broader project Amsterdam For All, which aims to help Amsterdam citizens with mobility issues to find the most accessible sidewalks, with respect to their specific needs. To cater to different people, this will be done through a highly user-dependent app. The focus of this research is on creating an interactive algorithm which can output only those paths the users are likely to prefer to take, given their specific accessibility needs, from and to their desired destinations.
Author: Alisa Todorova
This research was conducted by Alisa Todorova from the Department of Advanced Computing Sciences - Faculty of Science and Engineering, Maastricht University in collaboration with the AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Diederik M. Roijers and Shayla Jansen
Supervisors: Diederik M. Roijers, Shayla Jansen
Text simplification (TS) makes written information more accessible to all people, especially those with cognitive and/or language disorders. Despite much progress in TS due to ad- vances in NLP technology, the bottleneck issue of lack of data for low-resource languages still persists. To this end, we use a pivot-based approach to simplify Dutch medical and municipal text for the municipality of Amsterdam. This allows us to forego using a Dutch monolingual simplification corpus which, to our best knowledge, does not exist, in favour of using one in a higher resource language: English. We experiment with augmenting training data and corpus choice for this pivot-based approach. We compare the results to a baseline and an end-to-end LLM approach using the GPT 3.5 Turbo model. We find that, while we can substantially improve the results of the pivot pipeline, the few-shot end-to-end GPT-based simplification performs better on all metrics. With our work, we introduce a baseline for further comparison in the domain of Dutch municipal text and some improvements to the existing pivot-pipeline for simplifying Dutch medical text. Lastly, we provide a benchmark for comparing a pivot-based approach against an LLM approach.
This research was conducted by Daniel Vlantis in collaboration with AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Iva Gornishka
Supervisors: Shuai Wang & Iva Gornishka
The state of the quay walls in Amsterdam has deteriorated. In the past years, they have been monitored using tacheometry. The aim of this study is to forecast the time-series of the tacheometry data based on CPT and InSAR data. This was done by developing different configurations of BiLSTM models. The configurations conformed to three types of model fusion: early fusion, incremental fusion, and late fusion. These were compared to four baselines: a naive method and models taking input from each dataset separately. The models were compared based on their RMSE, MAE, and MASE scores. Incremental fusion yielded the significantly best results both on itself and in combination with late fusion. Early fusion did not benefit the performance. This is in line with previous studies using incremental and late model fusion, and contrary to a study using early fusion to predict deformation.
This research was conducted by Julian El-Fasih in collaboration with the Bridges and Quay Walls Program (Programma Bruggen en Kademuren) and the AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Pantelis Karamitopoulos
Supervisors: Frank Nack & Pantelis Karamitopoulos
This thesis addresses existing gaps in Long Form Question Answering (LFQA) research, specifically focusing on the Dutch language, which has been identified as a low-resource language regarding available datasets and conducted experiments for QA tasks. The study additionally positions LFQA research in a domain-specific setting which is the municipal domain. We scrutinize LFQA through a commonly used two-step pipeline involving information retrieval and answer generation, evaluated separately and in conjunction. We apply an experiment which substitutes retrieved-context documents with random ones to ascertain the impact of retrieval. The quality of answers is assessed quantitatively using ROUGE and BERT-score metrics, established evaluation methods from language translation studies. Moreover, a qualitative analysis complements the quantitative data to examine the representativeness of our two-step pipeline’s results. With these approaches, we not only tackle the urgent need to scrutinize the main LFQA components and the reliability of evaluation metrics but also provide a custom-made Dutch dataset to enhance resources for LFQA research in this language.
This research was conducted by Natali Peeva in collaboration with AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Iva Gornishka
Supervisors: Iva Gornishka & João L. M. Pereira
Reports made by citizens about issues in the public space, henceforth referred to as citizen reports, help contribute to the residential comfort of their cities by increasing the efficiency at which problems are identified and resolved. Some reporting systems, like the Dutch Signalen system (https://signalen.org/) have an automated classification component which does initial categorization. As a result of this users do not need to navigate subcategories and category-specific provisional information can be requested at the time of report submission.
In this work we investigate different aspects - feature use, multimodality, spatial context and data availability - which might be considered for the classification of citizen reports. We find that approaches which use pre-trained features in a more direct manner appear to perform better than those wherein more intermediate feature transformations are learned leading up to classification (e.g. through neural network layers). We compare multiple image representation groups, all including the same text representation in different Early Fusion schemes. We find that using our most general image representation, CLIP image without further additional representations works better than others as well as groups including additional representations. We expect this to be linked to a tendency towards overfitting. Our spatial-context models perform on par compared to our models which do not use this extra information. However, we find there are differences in precision and recall across report density. This leads us to suggest that using spatial context information might still be beneficial but should be obtained in a manner independent of and suggest some representations for this from the literature. The Meldingen production baseline, logistic regression based on TF-IDF text encoding, performs at the highest macro precision and does so disproportially to all other approaches. With respect to accuracy and macro recall, however, we find that a multi-modal CLIP representation using text and images results in the best outcomes with equal performance between models with and without spatial context. Both the baseline and our pre-trained representation approaches outperform the accuracy score reported in previous work.
This research was conducted by Reitze Jansen in collaboration with AI Team, Urban Innovation and R&D, City of Amsterdam and the Vereniging van Nederlandse Gemeenten (VNG).
Involved civil servants: Thijs Coenen & Iva Gornishka
Supervisors: Thijs Coenen & Stevan Rudinac
Vegetations are a vital part of our daily lives and are crucial for urban planning and the continuity of ecosystems. Accurate identification and delineation of individual trees in point cloud data is an important task in a variety of fields, including forestry, urban planning, and environmental management.
In this study, we have access to the Actueel Hoogtebestand Nederland (AHN) dataset, a publicly available high-resolution point cloud dataset generated by aerial laser scanning (ALS) throughout the Netherlands. The dataset that we will be using as the starting point for this study was previously labeled by the Amsterdam Intelligence team, with all tree points classified.
In this report, we present 2 different approaches for identifying individual trees. Both of the methods initial delineation is based on utilizing a KD-tree algorithm (Bentley, 1975) and vectors to make tree clusters. The first method identifies local maxima within a designated region through analysis of point density in that area. The second method uses some parameters to further split or merge the clusters.
A blog post summarizing the contributions can be found on amsterdamintelligence.com.
The related code is available here.
This research was conducted by Jorges Nofulla in collaboration with the department of Research and Statistics (Onderzoek en Statistiek, O&S) and the AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Nico de Graaff & Daan Bloembergen
Supervisors: Sander Oude Elberink, Michael Ying Yang, Nico de Graaff & Daan Bloembergen
A classification model that can assess the CEFR level of texts can be used to make websites more readable, to assess the quality of vacancy texts and to make news content, for example, information about the pandemic more accessible. This, in turn, can increase the level of civic participation and community involvement.
The aim of this study is to compare state-of-the-art deep learning-based models with traditional machine learning models that rely on handcrafted features. This research applies state-of-the-art deep learning based models BERT, RoBERTa and DistilBERT to perform CEFR text classification.
This research was conducted by Ayoub Abdelouarit in collaboration with AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Iva Gornishka
Supervisors: Maarten Marx & Iva Gornishka
Accessibility and safe movement in urban areas entail infrastructure that minimizes the risks for pedestrians and bikers with diverse levels of abilities. Recognizing and mapping unsafe areas can increase awareness among citizens and also inform city projects to improve their infrastructure.
This contribution presents an example in which the specific objective is to recognize the unprotected areas around the canals in the city of Amsterdam. This is accomplished through running image processing algorithms on 11K waterside panoramas taken from the city of Amsterdam open data portal. We created an annotated subset of 2K processed images for training and evaluation. This dataset debuts a novel pixel-level annotation style using multi-lines, which is found to generalize better than traditional polygon-like bounding boxes.
To determine the best inference practice, we compared the IoU and robustness of several existing segmentation frameworks. The best method achieves an IoU of 0.79. We discuss the broader application of the presented method for the problem of “transparent object detection in urban context” and show that it is feasible.
This research was conducted by Jorrit Ypenda in collaboration with AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Maarten Sukel & Cláudia Pinhão & Daan Bloembergen
Supervisors: Maarten Sukel & Hamed S. Alavi
Micromobility vehicles, such as bicycles, are often left on the sidewalk, where they limit the space of the already narrow pedestrian zone. A better understanding of micromobility parking and the possibility to predict the demand is needed to improve the management of these facilities and ultimately to prevent the obstruction of public space. Previous research was mainly focused on the parking of other vehicles, such as cars, introducing a lack of research and data related to micromobility parking. Therefore, this research aimed to use historical counts of the number of parked micromobility vehicles along with neighborhood characteristics to analyze and predict the parking occupancy on the sidewalk.
To achieve this goal, both supervised and unsupervised machine learning techniques were applied. Tree-based ensemble models proved to be suitable for predicting parking occupancy. In terms of predictive features, historical observations were the most influential predictor. The inclusion of the cluster results and neighborhood variables such as land use and the presence of points of interest further improved the predictions. Furthermore, clustering has made it possible to summarize multivariate information and to identify areas of similar characteristics.
This research was conducted by Johanna Fiebag in collaboration with AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Shayla Jansen & Cláudia Pinhão & Lino Miltenburg
Supervisors: Shayla Jansen & Frank Nack
Root systems of trees are hard to monitor, while it is essential to know where and how they grow for city development and to maintain the urban trees healthy. We created a model that estimates how much root volume trees need now and in the future and converts this volume to 3D cylinders to include them in 3D Amsterdam.
We try out three methods that vary in their required input parameters: the static, tree dictionary, and tree growth methods. We investigate if the model can provide accurate and valuable insight into the location and the impact of tree roots and validate and compare the three methods for three diverse subregions in Amsterdam: het Wallengebied, IJburg and Sarphatipark. We did this based on predictive capabilities, ground radar scans of real root systems, and damage reports involving roots.
We found that the model is capable of giving a general indication of the space that the roots require but that, in some cases, it is necessary to reshape the cylinders to provide a more accurate indication of the location of the roots. In addition, we found that the model can be used to investigate which areas are prone to root lifting in the future. The three methods perform similarly in estimating the current root volumes. However, the static and tree growth method are expected to respectively under- and overestimate the future root volumes. Therefore, we used the tree dictionary method to estimate the root volumes that the trees need to stay healthy for the whole city of Amsterdam.
Related Blog Post: here
Corresponding Code Repository: here
This research was conducted by Iris Reitsma in collaboration with the 3D Amsterdam team of the City of Amsterdam.
Involved civil servants: Wietse Balster & Nicole Archangel
Scientific Supervisor: Nicole Archangel & Dr. Robert Belleman
This work focuses on determining the entrance (or outside) accessibility of buildings by detecting and segmenting individual steps in point clouds to provide people with reduced mobility with information on the number of steps, step heights, and step widths for each address in Amsterdam. This will make it easy for PRMs to find if an address is accessible to them and additionally will provide the local government of Amsterdam an overview of the extent of entrance accessibility issues in the city.
This research was conducted by Jeroen van Wely in collaboration with AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Daan Bloembergen & Cláudia Pinhão.
Supervisors: Cláudia Pinhão & Shaodi You
Persons with reduced mobility (PRM) have difficulties determining whether they can access venues, due to inadequate information about venues’ accessibility. Previous research has largely used computer vision techniques for this purpose and general accessibility-scores have been provided. However, due to the variety of PRM, both approaches lack user experiences that can provide detailed information on venue accessibility. Existing research has not yet investigated online reviews to obtain accessibility-related information. Therefore, we will apply text analysis to online reviews for information provision about accessibility.
The contributions of this research are three-fold. First of all, we provide an overview of information from user experiences about the accessibility of venues for PRM, as opposed to a general score or assessment. Secondly, we apply an approach to information provision for venue accessibility that has not yet been used. Namely, text analysis with NLP techniques, as opposed to the previously implemented computer vision techniques on images. Thirdly, we take multiple existing models that have been utilized for online review analyses and apply them to a new domain of reviews. We will determine what models perform best for accessibility texts.
This research was conducted by Lizzy Da Rocha Bazilio in collaboration with AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Iva Gornishka & Cláudia Pinhão.
Supervisors: Iva Gornishka & Clemencia Siro
Many Europeans experience difficulties with reading. This can cause them to be unable to take in important information that is meant for them. For governments and institutions, it is thus important to write their texts in a way that makes them most accessible. The words that are used in these texts play an essential role in their perceived complexity. Automatic simplification of these complex words is called lexical simplification and can help increase the accessibility of texts.
Previous work on lexical simplification has relied mostly on lexical resources. The main limitations of such approaches are 1) that they cannot generate all simplifications for all complex words, and 2) that the simplifications are solely based on the complex word, not the sentence around it. This can result in unsuited simplifications. Therefore, a model for contextualized lexical simplification was introduced by Qiang et al. (2020), which does not rely on lexical resources and generates simplifications based on the context. It makes use of the contextual language model BERT to generate substitutions of the complex word. This approach achieves state-of-the-art results on common benchmarking datasets.
In this work, experiments are performed to increase the performance of this method by fine-tuning the model toward simple language generation, this way the model will produce simplifications rather than substitutions. Moreover, the possibility of fine-tuning to adapt the model to the domain of Dutch municipal texts is explored. To do so, first, a dutch variant of LSBert, called LSBertje is developed. The model is then fine-tuned to produce domain-specific simplifications. The findings of this work underline the adaptability of these contextual language models: exposure to simple language is seen to yield simpler simplification candidates. Domain-specific simplifications were not achieved, but the findings suggest that fine-tuning for domain-specific simplifications is a feasible research angle for future work.
This research was conducted by Eliza Hobo in collaboration with AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Iva Gornishka & Cláudia Pinhão.
Supervisors: Iva Gornishka & Lisa Beinborn
In many governmental organizations, the necessity to regain control of document archives exists. Different laws (such as the Archiefwet 2021) determine what information should be managed, and how, and what documents need to become publicly available. Together with the Municipality of Amsterdam, we explored their archival problem and how Natural Language Processing can assist in solving it.
The biggest challenge in training a model for labeling assistance is the lack of quality labeled data and the large set of imbalanced classes. The texts are all in Dutch, generally written formally, and tend to include domain jargon. This means at least part of the data exists in a low-resource domain; it is neither English nor part of the standard training data of typical language models. On top of that, the classes are imbalanced; they are very unequally represented.
In this work we explore the use of Active Learning (a method aiming to reduce the labeling burden by selecting more informative samples for labeling) for the task of multi-class classification of
documents in an imbalanced low-resource setting.
Furthermore, to help future research in this field and ease result comparison, we present a dataset for the task of low-resource classification of dutch legal documents at two levels of imbalance.
This research was conducted by Emiel Steegh (LinkedIn) in collaboration with AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Ymkje Galama & Iva Gornishka.
Supervisors: Ymkje Galama & dr. Giovanni Sileno
Image datasets captured from public spaces are used in many applications and are especially crucial for computer vision tasks requiring real-world data. However, these datasets pose an inherent risk to the people appearing in the images and are often subject to strict privacy regulations that dictate their use and distribution. Through image anonymization, which aims to remove the identifiable aspects of people from images, we can mitigate the privacy issues associated with image datasets, allowing them to be freely shared for collaboration, future research, and peer review.
In this work, we present our research on methods of generating and evaluating realistic anonymized image datasets that can be used in a wide range of applications. We use conditional Generative Adversarial Networks to develop models for generating anonymized people in place of the identifiable people who appear in the original images. Furthermore, in the absence of an industry-standard evaluation method for person anonymization, we also propose anonymity and diversity metrics as part of a comprehensive method for evaluating the anonymity and realism of generated anonymized image datasets.
This research was conducted by Kaleigh Douglas in collaboration with AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Laurens Samson & Iva Gornishka.
Supervisors: Laurens Samson & Phillip Lippe
One of the goals of the municipality of Amsterdam is to improve the livability of the city. This includes affordable housing for everyone and the limitation of nuisance caused by tourists. Houses that are used as illegal hotels, reduce the number of houses available to live in. Moreover, the increased number of tourists in the middle of residential areas is causing nuisance. Therefore, the municipality is committed to reduce
the negative impact of vacation rentals and has set up a number if regulations in which vacation rental is illegal.
This work explores the use of Heterogeneous Graph Attention Networks to predict illegal vacation rentals with the aim of aiding enforcers in their usual process of investigating houses and determining whether fraud was conducted or not.
This research was conducted by Petra Ormel in collaboration with AI Team, Urban Innovation and R&D, City of Amsterdam and the department of Housing, City of Amsterdam.
Involved civil servants: Ammar Khawaja & Iva Gornishka.
Supervisors: Ammar Khawaja, dr. Stevan Rudinac & Ujjwal Sharma
One of a municipality’s responsibilities is to regularly monitor suspended streetlights and cables for a safe and reliable environment. Mobile laser scanned 3D point clouds offer a great potential for creating high-precision digital representations of assets in street scenes. The goal of this thesis is to develop a computationally efficient and reliable pipeline that automatically detects suspended streetlights in urban scene point clouds.
In this thesis, we discuss the relevance of the problem to the municipality, review related work in object extraction from point clouds, present and describe our four-staged pipeline for the extraction of suspended streetlights and cables, and provide an extensive evaluation in terms of performance and applicability.
This research provides four contributions to the field of point cloud processing: a robust cable extraction algorithm. Second, a cable type classifier, an algorithm that can detect cable attachments, and a labelled training dataset for supervised machine learning models.
This research was conducted by Falke Boskaljon in collaboration with AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Daan Bloembergen & Chris Eijgenstein.