This research was conducted by Natali Peeva in collaboration with AI Team, Urban Innovation and R&D, City of Amsterdam.
Involved civil servants: Iva Gornishka
Supervisors: Iva Gornishka & João L. M. Pereira
MSc Thesis - UvA - Automatic Answering of City Council Questions - Natali Peeva
MSc Thesis by Natali Peeva
This thesis addresses existing gaps in Long Form Question Answering (LFQA) research, specifically focusing on the Dutch language, which has been identified as a low-resource language regarding available datasets and conducted experiments for QA tasks. The study additionally positions LFQA research in a domain-specific setting which is the municipal domain. We scrutinize LFQA through a commonly used two-step pipeline involving information retrieval and answer generation, evaluated separately and in conjunction. We apply an experiment which substitutes retrieved-context documents with random ones to ascertain the impact of retrieval. The quality of answers is assessed quantitatively using ROUGE and BERT-score metrics, established evaluation methods from language translation studies. Moreover, a qualitative analysis complements the quantitative data to examine the representativeness of our two-step pipeline’s results. With these approaches, we not only tackle the urgent need to scrutinize the main LFQA components and the reliability of evaluation metrics but also provide a custom-made Dutch dataset to enhance resources for LFQA research in this language.