User study methodology

We conducted a user study with two participant groups: (1) hearing parents of DHH children, and (2) NGT teachers who teach hearing parents. Although the broader project included both semi-structured interviews on parents’ experiences with sign language learning and a user study on the ZINinNGT prototype, this paper focuses only on the latter. For a detailed account of parents’ experiences with sign language learning, see [16]. (Note that part of this work has been discussed in the introduction.)

Participants

Eligibility and recruitment

Hearing parents of DHH children were eligible to participate if they were currently enrolled in or had previously completed an NGT or Sign-supported Dutch (NmG) course. NmG combines spoken Dutch with key-word signing, aiming to make spoken language more visible.¹ Both NGT and NmG learners were included because, in the Netherlands, hearing parents of DHH children are generally eligible for health insurance-covered parental sign language courses, but in some regions these courses teach NmG rather than NGT. Therefore, including NmG learners allowed us to increase our participant pool and achieve greater geographic diversity. There were no restrictions regarding the age of the participants’ children. Although ZINinNGT is designed for parents of children aged 1 to 5, we wanted to increase our participant pool and believed that parents’ reflections on their past learning experiences could provide valuable insights into what they would have needed in an NGT app.

NGT teachers were eligible to participate if they had experience teaching NGT to hearing parents of DHH children. There were no restrictions regarding the duration of teaching experience or hearing status. It is, however, important to note that the World Federation of the Deaf emphasizes that Deaf people must stand at the forefront of sign language teaching, especially when teaching DHH children and their families [35]. In practice, parental sign language courses in the Netherlands are taught by both DHH and hearing instructors, regardless of whether the course focuses on NGT or NmG. This reflects the fact that the Netherlands has one official teacher training program for NGT (a four-year bachelor’s program²), which is open to both hearing and DHH students, with hearing students making up the majority of graduates.

Recruitment took place through social media sites aimed at hearing parents of DHH children, as well as through organizations serving parents of DHH children, such as the Royal Dutch Auris Group, the Dutch Federation of Parents of Deaf Children (FODOK), the mental health and social services for the DHH (GGMD) and Royal Dutch Kentalis, and personal contacts. Participants received a 20-euro gift card upon participation.³

Parent demographics

A total of 22 parents took part in this study. However, one parent’s data were excluded from the analysis because their child’s age differed substantially from that of the other participants, by 12 years above the mean. This left a final group of 21 parents (17 mothers and 4 fathers), comprising 20 different families. The mother and father from the same family tested the prototype together, while all other participants took part individually.

Our sample was geographically diverse, with participants living in seven provinces across the Netherlands, representing the southern, central, and northwestern regions. Most parents had one DHH child, while one parent had two. The children’s ages ranged from 2 to 9 years, with a mean age of 6 years. All participating parents were hearing, except for one parent who was hard of hearing. Although the study targeted hearing parents, this parent was included because they had no prior experience with sign language before learning that their child was DHH. Finally, parents were asked about their prior experience with educational apps. Thirteen parents indicated that they had used such apps to some extent. A summary of the parent demographics can be found in Table 4.

 | Total
Gender
Female | 17
Male | 4
Age of DHH children (years)
Mean (SD) | 5.57 (2.31)
Median [min, max] | 6.00 [2.00 – 9.00]
Hearing Status
Hearing | 20
Hard of Hearing | 1
Type of Course(s) Taken
NGT | 9
NmG | 7
NGT & NmG | 5
Prior Experience with Educational Apps
Yes | 13
No | 8

Teacher demographics

Six NGT teachers (four female, two male) took part in this study, and all participated individually. Three teachers identified as hearing, and three identified as Deaf. Two of the Deaf teachers considered NGT their first language. The teachers in our sample were rather experienced, with a mean of around 15 years of NGT teaching experience (range: 3–27 years), including teaching groups other than parents. Lastly, all but one teacher reported having used educational apps either currently or in the past. Table 5 summarizes the teacher demographics.

 | Total
Gender
Female | 4
Male | 2
Hearing status
Deaf | 3
Hearing | 3
First language(s)
Dutch | 3
Dutch & Frisian | 1
Dutch & NGT | 1
NGT | 1
Teaching experience (years)
Mean (SD) | 14.83 (8.23)
Median [min, max] | 14.50 [3.00–27.00]
Prior Experience with Educational Apps
Yes | 5
No | 1

Testing procedure

This study was carried out by the first author between February and May 2024 and involved a think-aloud procedure. Sessions took place at locations chosen by the participants—typically parents’ homes and teachers’ workplaces. The first author is hearing and a native speaker of Dutch, who learned NGT as an adult and uses it daily in professional contexts. The study was conducted in Dutch, except for two sessions that were conducted in NGT and one in NmG. All sessions were audio-recorded, and NGT sessions were additionally video-recorded.

For the NGT sessions, an NGT-Dutch interpreter was present, chosen according to the participant’s preference. Throughout the study, the researcher and participant communicated directly in NGT with the interpreter providing simultaneous translation into Dutch. This setup allowed the researcher to verify her understanding without interrupting the flow of direct NGT communication. Although the presence of an interpreter influences interactions,⁴ it was considered important to minimize the risk of miscommunication or missing important details in participants’ responses.

Participants were given a smartphone with the ZINinNGT prototype opened on its home screen. They were asked to explore the prototype, click through all available buttons, and share their thoughts and actions throughout the process. The researcher only asked clarification questions during this stage. If the researcher noticed by the end that certain features had been overlooked, the participant was asked to explore those as well.

After testing, follow-up questions were asked, including whether participants would use the app once it became publicly available. Additional questions concerned future additions, such as a voice-over and videos showing conversations in NGT. As these questions were highly suggestive (e.g., “Would you use [feature] if it were implemented?”) and specific to the ongoing development of ZINinNGT rather than to sign language learning apps more generally, they are not included in the analysis presented in this paper.

Coding and analysis

All sessions were transcribed automatically using Whisper AI software. Transcripts of the Dutch and NmG sessions were manually checked against the original audio recordings, while transcripts of the NGT sessions were verified against the corresponding video recordings. Once verified, all transcripts were imported into ATLAS.ti for coding.

Feedback was coded thematically using four broad categories: user interface, content, scope, and intended use. Within each category, additional focused codes (e.g., video looping, native signers) were applied to capture the specific aspects participants mentioned.

Footnotes

1. However, in practice, NmG often fails to represent either Dutch or NGT clearly, making it difficult to understand for those who rely primarily on visual input [33, 34]. ↩

2. See Inhoud & vakken Leraar Nederlandse Gebarentaal/Tolk NGT—Voltijd | Hogeschool Utrecht for more information on the teacher training program NGT [36]. ↩

3. This excludes three teachers who were contractually required to participate in scientific research. ↩

4. See for example [37] on interpreter influences. ↩