Paintings, manuscripts, photographs, videos, newspaper articles - heritage institutions have an enormous wealth of digitized collections. Artificial Intelligence (AI) plays an important role in analyzing these collections and making them accessible. CWI, the KNAW Humanities Cluster, the Dutch National Library, the Netherlands Institute for Sound and Vision, the Rijksmuseum, TNO, the University of Amsterdam and the Vrije Universiteit Amsterdam therefore join forces in the Cultural AI Lab.
Together they will use the possibilities of AI for cultural research and make AI technology aware of the cultural context. The research within the lab aims to make AI technology more aware of cultural contexts and also to align cultural institutions with state-of-the-art AI research in the Netherlands.
Five research projects
The parties will work closely together to develop AI tools that can be applied in the cultural heritage sector. In the coming months, the first five research projects will start, including an investigation into the automatic tracing of colonial terminology in collection data and an investigation into framing in online journalism.
Multiple perspectives on heritage
“The collaboration also opens up possibilities to investigate how AI can help to map out multiple perspectives on heritage”, says Laura Hollink, researcher at the CWI. “Our goal is to be able to create AI systems in the future that can deal with the rich, subjective, polyphonic data from the cultural heritage sector.”
Cultural AI
The lab builds a bridge between cultural heritage institutions, humanities and computer science. Data and information from relevant heritage institutions and technical knowledge from research institutes form the basis for the development of AI tools that do justice to the complexity of human culture and can be applied within the cultural heritage sector in the Netherlands. All research projects within the lab are related to this.
Current and future research within the Cultural AI initiative will be divided among the institutions involved. PhD students and postdocs will initially spend four years working on research within the heritage institutions involved.
The Cultural AI Lab will start with five research projects:
AI:CULT Culturally Aware
Automated analysis and enrichment of object descriptions in museum collections using AI. The goal is to use artificial intelligence within the cultural heritage in a transparent and inclusive way, while keeping the user in control and providing insight into changes made inspired by AI research.
KNAW Meertens Institute, CWI, KNAW Humanities Cluster, Netherlands Institute for Sound and Vision and Dutch National Library. Funded by the Dutch Research Council.
SABIO - the SociAl Bias Observatory
Automated analysis of collection descriptions in museum collections in order to detect colonial terminology. This research maps the cultural bias in collections and updates the information without overwriting existing collection data.
KNAW Humanities Cluster, Museum of World Cultures, Netherlands Institute for Sound and Vision and Dutch National Library. Funded by Network Digital Heritage
Better Informing Citizens about Current Debates
Automated analyses of contemporary debates in the press with the aim of developing a tool that makes it possible to evaluate and improve the quality of online debates.
KNAW Humanities Cluster and Tilburg University
RE-FRAME
Analysis of context and framing in online journalism. In RE-FRAME we investigate the reuse of resources and the role they play in the construction of audio-visual journalistic storytelling through content analysis and (action-based) production analysis. We investigate how new technologies, such as Automatic Speech Recognition and Computer Vision, contained in the CLARIAH Media Suite, can play a role in finding and interpreting content and in journalistic practice.
Utrecht University and Netherlands Institute for Sound and Vision
Researcher in Residence Programme Cultural AI
External researchers can use the data from the Dutch National Library to propose research on the themes of the Cultural AI Lab. The selected researchers will receive compensation and can make use of the facilities of the library during their research. In 2021, the library will collaborate with researcher Simon Kemper (Leiden University) on a project to locate entities (persons, places and organizations) in the multilingual colonial digitized newspapers using various AI language models.
-
Article
New collaboration develops AI for cultural heritage
CWI starts a new collaboration with heritage and research institutions around artificial intelligence (AI). In the Cultural AI Lab they will jointly explore the possibilities of AI for cultural research and raise AI’s awareness of the cultural context.
-
Article
AI:CULT - Culturally aware AI
Automated analysis and enrichment of object descriptions in museum collections using AI. The goal is to use artificial intelligence within the cultural heritage in a transparent and inclusive way, while keeping the user in control and providing insight into changes made inspired by AI research.
The AI:CULT project addresses the gap between AI and our digital cultural heritage. Cultural heritage data is rarely objective data. The very reasons for certain heritage data to be preserved, its interpretation throughout time, and the way heritage data is accessed after digitalisation is all subject to strong biases. The inherent richness, subjectivity and polyvocal nature of cultural heritage data limits and often even rules out the responsible use of AI. How do we model that “Seventeenth Century” and “The Golden Age” refer to the same era, yet are not fully synonymous and carry different semantic payloads? Current state of the art AI cannot deal with these subtleties in a way that does justice to the important role of the heritage institute as a trusted source of information. Thus, the heritage sector is under threat to be left out of the current global success of AI. AI:CULT will allow heritage institutes to use AI in ways that align with their role in society: transparent, inclusive, and keeping the user in control.
The project addresses two case studies with societal parties tasked with providing access to national heritage, and who have voiced their vested interest in using AI for their workflows: the National Library (KB) and the Institute for Sound and Vision (NISV): (i) automatically analysing and enriching object-level descriptions and (ii) creating data stories and narratives from raw collection data. Both institutions acknowledge that the straightforward application of AI reflects biases present in the training data. In the AI:CULT project bias detection and filtering methods will be developed that will be directly tested on the heritage institutions’ workfloors.The two Ph.D. students on this project will be based at KNAW Humanities Cluster and CWI in Amsterdam.
Source: Cultural AI - AI:CULT
-
Article
BETTER-Mods - Better Informing Citizens about Current Debates
Automated analyses of contemporary debates in the press with the aim of developing a tool that makes it possible to evaluate and improve the quality of online debates.
This project aims to develop and evaluate digital tools for improving the quality of online discussions about current debates (ranging from, for example, COVID-19 to Black Pete and climate change), to better inform citizens. In particular, we will (1) analyse discussions in separate domains and at several points in time, and develop computer-assisted moderation models that are optimally catered to the needs of human discussion moderators; (2) develop automatic forum summarisation techniques that can distinguish between different viewpoints and arguments put forward in a discussion; and (3) develop interactive visualisations of the generated summaries, with an overview of the topics and viewpoints, to facilitate exploration of debates.
Source: Cultural AI - BETTER-Mods
-
Article
RE-FRAME-Tracing Re-Use of Audiovisual Data in Journalism
Analysis of context and framing in online journalism. In RE-FRAME we investigate the reuse of resources and the role they play in the construction of audio-visual journalistic storytelling through content analysis and (action-based) production analysis. We investigate how new technologies, such as Automatic Speech Recognition and Computer Vision, contained in the CLARIAH Media Suite, can play a role in finding and interpreting content and in journalistic practice.
This PhD-project focuses on the practices of re-use and framing of audiovisual data within journalism, by exploring how digital tools and AI techniques like Automatic Speech Recognition (A.S.R.) and Computer Vision (C.V.) play a role in the retrieval, selection, interpretation and storytelling processes. Through content analysis, interviews and action-based production analysis with journalists, I will explore the modalities of A.I. driven re-use and develop digital methods based on tool- and data criticism that can be employed in journalistic practice and training.
Source: Cultural AI - RE-FRAME
-
Article
SABIO - The SociAl BIas Observatory
Automated analysis of collection descriptions in museum collections in order to detect colonial terminology. This research maps the cultural bias in collections and updates the information without overwriting existing collection data.
The SociAl BIas Observatory (SABIO) project is aimed at investigating bias in the digital collections of the members of the Dutch Digital Heritage Network. In this project, we investigate how collection managers and curators create and add metadata to collection objects, and how bias in these metadata can be detected using statistical models. We aim to create a knowledge graph on top of existing collection databases that makes prejudices and imbalances in the data explicit such that they can be addressed, as well as taken into account by users of the data.
Source: Cultural AI - SABIO