Increasing understandability and reducing the negative impact of low literacy in citizens’ lives deeply aligns with our Amsterdam for All goals to measure and improve the accessibility of our city. Therefore, we have decided to start researching the topic by facilitating a 2-day workshop that resulted in a better definition of the challenges, compilation of ideas, and definition of a work direction. We set out to answer the following question: How can we support civil servants from the City of Amsterdam in their process so they can deliver better accessible content?
In this blog post, we are going to share the outcomes of our workshop and we will walk you through the challenges, processes, and solutions. While some of these solutions are AI-related, others are about improving processes. In any case, we hope all of them inspire you to improve readability through your own skills and expertise.
The workshop team
To better understand the challenges and needs of writers from the City of Amsterdam, we invited colleagues with diverse expertise from different teams and departments. These included Frank Willemsen and Stefan Baas - advisors from the departments of services and communications where our editors put a lot of effort into ensuring that our policy, services, and important information are communicated to all citizens in an understandable, accessible and trustworthy way.
We also invited communication advisors, editors, and copywriters who help other departments with editorial tasks - Hanna Rab, Timo Nieuwenhuis, and Jason Fiawo. Finally, Aartie Mahesh, Jeffrey Boxem, Iva Gornishka, and Banoe Abdullah helped us understand more about the struggles of communicating technical details about specific projects to the citizens of Amsterdam. Our UX designer, Cláudia Pinhão, facilitated the workshop and made sure that everybody's voice was heard, recorded, and turned into useful insights.
The process
At the end of December 2021, we have facilitated two remote workshop sessions of 2 hours each. We used an adapted version of the 4C’s framework from Jonathan Courtney, with the following steps:
- Collect - understand current processes, and define challenges and opportunities
- Choose - choose what to focus on
- Create - generate multiple potential solutions for the chosen focus area
- Commit - create a plan of action and responsibilities to make it happen
Miro and MS Teams were used to facilitate these remote workshop sessions
This is the suite of remote collaboration tools we used during and after our sessions:
- Miro: virtual whiteboard for all workshop exercises, sticky notes, and drawings
- Airtable: spreadsheet-database hybrid for documenting all quotes and insights
- MS Teams: chat-based collaboration platform for video conferencing
- Google Slides: online presentation app for sharing results
Collaboration tools used during and after our workshops
Collect: Mapping the current processes and challenges
In the first session, we did several exercises, such as mapping the process of writing accessible texts, and an adaptation of the sailboat, which helps identify different challenges and opportunities. During each exercise, we had 10 minutes for answering the questions and around 20 minutes for sharing them and debating about it. From this session, we could segment our main archetypes - we called them Egalitarian Writer, Technical Writer, and Field Expert.
Writer archetypes discovered during our workshop
The criteria that set apart the archetypes are the type of expertise, writing skills, common language skills, target audience, and project goals. These archetypes help us to understand the differences between profiles, identify who fits into each category, and what the organization is lacking.
Next, we tried to align and map all the writing processes. From the results, we have identified some differences and gaps. Fortunately, the individual workflows complement each other and can be compiled into a single map depicted below:
Optimal writing process map
After mapping the processes, we did the sailboat exercise, where participants wrote down the main challenges and opportunities they perceived during their process. Below, we explain each step of the process map, together with a short summary of the insights from the participants:
Define
The workflow starts by answering the right introductory questions to reduce mistakes and avoid the need to rewrite the text further in the process. The questions involve defining the message, audience, medium, length, context, call to action, and stakeholders. A key challenge is that writing about diverse topics to a broad audience (such as the citizens of Amsterdam) makes determining all requirements difficult - there is simply not one size that fits all.
Research
The following step is conducting online research, interviews with colleagues or citizens, and collecting relevant documentation from projects. The overload of available information and the lack of standard form(s) for collecting project details can slow down the whole process or cause many back and forth iterations
Write
Create a content table, write down keywords and the concept. After review and testing, rewrite the text based on the remarks and the new information. To ensure the readability of the text, one needs to “kill their darlings” - that is, let go of anything that is not necessary to share the core message. Also, even though clear and accessible language (A2/B1) is generally preferred and fits everyones’ needs, it was mentioned that there are not enough writers within the organization who are experts in these levels.
Review
Use online tools to do a self-review or send the text to colleagues for peer review. Our workshops showed that this is one of the most crucial steps in the process and it could easily turn into a bottleneck if we don’t make sure that colleagues with a fresh perspective are available to review the text.
Test
Find a group that fits your target audience and test if the information is understandable and actionable. While this can be costly, time-consuming and sometimes skipped, there are research companies that already collaborate with the municipality, such as ABC Foundation.
Approve and Publish
Get a final approval from the project team and/or others, depending on the project and publish it in the chosen medium.
Evaluate & Report
Evaluate if the target audience understands the message and knows what to do. Report the learnings to inform next projects.
Choose & Create
During the second session, we did lightning demos and quick ideation exercises. Here, we would walk you through the four main questions we tackled and some of the corresponding proposed solutions:
How might we set up the right environment to increase the quality of the writing process?
Writing clear and accessible language (A2/B1) means having the right resources, such as experts in the field, but also setting up the right mindspace to write and knowing precisely in which level to write.
How might we improve the review process?
The ideas generated point in two main directions. First, automated self-review tools could help our colleagues better reflect on their own texts and avoid common mistakes, such as complex words or difficult sentence structure. Furthermore, an online collaborative tool could be a place where writers share their texts for peer-reviewing within the organization.
How might we improve the test process?
Improving the testing process requires more time and budget, but it also provides an easy way to connect with the target audience. We ideated about having citizens and children’s panels, but also an invitation app, or a feedback button on our websites.
How might we present a text that fits a broad audience?
Even though recent studies discovered people prefer reading text in common language, some use cases make it difficult, such as legal or academic text. Therefore, the ideas shared in this question explore the possibilities of not changing the text, but introducing new ways of communication to make it more understandable and accessible, such as videos and images, but also creating other text versions with automatic language translations into other levels.
Commit: next steps
At the end of the second session we voted, and the possible tool that we called ‘Amsterdamly’ was the most voted idea.
A tool that scans your text (like auto-correct) and gives tips about difficult words, sentence length, and structure. This tool is meant to identify the problems in the text and suggest alterations based on the language level and audience you are targeting.
Our target for the Amsterdamly tool will help provide A2/B1 levels for important communications. We will set up a new workshop to collect requirements and afterwards make a competitor's analysis. Our goal is to advise on an already existing tool that fits the requirements, or if there is no good fit, start developing a tool ourselves.
We are happy to hear your thoughts on our research, also if you have been working on similar topics that overlap with the solutions we shared in this blog post. We encourage you to get in touch with us!
More about the topic and related tools
[1] Fact and numbers about low literacy by Stichting Leven en Schrijven: https://www.lezenenschrijven.nl/voor-gemeenten/feiten-en-cijfers
[2] Research into low literacy conducted by Gemeente Amsterdam: https://onderzoek.amsterdam.nl/publicatie/nederlandstalige-laaggeletterden-in-amsterdam
[3] Why it is important to write in an understandable manner, as well as practical tips and guidelines for clear language (Schrijfwijzer Gemeente Amsterdam): https://www.amsterdam.nl/schrijfwijzer/waarom-belangrijk/
[4] User research company with low literacy citizen panels:
https://a-b-c.nu/
[5] An overview of existing readability tools by Didaktiek Nederlands: https://didactieknederlands.nl/handboek/2021/10/wat-zijn-tekstbegrijpelijkheids-voorspellingen-waard-een-vergelijkend-onderzoek-deel-4/#lint
Technical Resources
[6] Lexical resources by Université catholique de Louvain for foreign language (L2) learning, teaching and research graded following the Common European Framework of Reference for Languages (CEFR), available in 6 languages: https://cental.uclouvain.be/cefrlex/
[7] Papers and code repositories related to readability and text simplification: https://paperswithcode.com/search?q_meta=&q_type=&q=CEFR
[8] Kleijn, Suzanne. Clozing in on Readability. Netherlands Graduate School of Linguistics, 2018. A dissertation on readability: https://dspace.library.uu.nl/handle/1874/363346
[9] Martinc, Matej, Senja Pollak, and Marko Robnik-Šikonja. "Supervised and unsupervised neural approaches to text readability." Computational Linguistics 47.1 (2021): 141-179. An overview of traditional and recent approaches, and resources for text readability: https://direct.mit.edu/coli/article/47/1/141/97334/Supervised-and-Unsupervised-Neural-Approaches-to
Authors: Iva Gornishka & Claudia Pinhao.
Source: Amsterdam Intelligence