A minimal desktop application to make learning more accessible for deaf or hard of hearing students in mainstream classrooms

A minimal desktop application to make learning more accessible for deaf or hard of hearing students in mainstream classrooms

Context

  • Academic project for the course - Inclusive Design for Accessible Technology.

  • I was a researcher for this project and focused on secondary research, stakeholder interviews, persona, and research synthesis. I also conduced an accessibility audit for the report and designed the mid-fidelity prototypes.

  • The team consisted of 3 members and spanned 14 weeks.

What were we trying to solve?

Classroom accommodations help in making learning accessible for deaf/Hard of Hearing students. However, most accommodations are designed to support students with profound or severe hearing loss, leaving room for addressing the needs of students with mild, moderate, or even situational hearing loss. This led to our research question -

How can we make mainstream classrooms more accessible for the Deaf/HH community, with a focus on students with mild to moderate hearing loss?

Our solution - Knowl

Knowl is a minimal desktop application that can be easily integrated into the classroom learning environment.

Research

Our process began with the research phase. To understand the problem space better, we conducted both primary and secondary research.


Secondary Research

We conducted a literature review to understand existing solutions in building inclusive classrooms for deaf/HH students. This phase of our research helped us develop a better understanding of the following:

Accommodations in mainstream classes

Accommodations

in universities for deaf students. eg: Gallaudet

Challenges with existing accommodations and accessibility aids

Accessibility needs based on the severity of hearing loss.

We then categorized our findings into clear sections to facilitate collaborative problem-solving. A snapshot of our secondary research is provided below:

Primary Research

We wanted to strengthen our secondary research by gathering information directly from our users and experts. We conducted interviews with the following participants.


a. A graduate student pursuing a technical program in a mainstream university with moderate hearing loss. View the interview script

b. An educator and campus accessibility expert specializing in inclusive education.

What did we find?

We used affinity mapping to uncover themes and synthesize our data.

An initial round of analysis,generated 13 categories of user issues which we grouped into three broad themes:

Missing Information

Inspite of using accessibility aids like hearing aids, students with mild/moderate hearing loss often periodically missed information in a lecture or discussion. It was also difficult to seek out this missed information in a classroom setting.

Social Disconnect

Incidental learning and informal discussions outside of regular classroom discussions are overlooked facets of engaged learning which could create a sense of disconnect or isolation for students with mild/moderate hearing loss.

Everyone is unique

Figuring out an accessibility aid that worked for them was often a challenging experience for our users. In addition to considering factors such as the severity of their hearing loss, users also took into account their personal preferences ranging from comfort to dressing style.

After carefully considering the themes, we decided to center our solution on the first theme - Missing Information, as it was most within the scope of a semester-long project.


The insights generated from the analysis helped us focus on the three main facets of missing out on information in a classroom setting. This helped us come up with our three main "How Might We" statements to shape our design process:

How might we?

How might we reduce the barrier to accessing contextual/oral information missed out in the classroom?

How might we reduce the dependence on positioning/seating in the classroom for effective listening and participation?

How might we reduce the effort to consume information in group environments to enable active participation?

How might we?

Crafting a persona

With the HMWs in place, we decided to build a persona to conceptualize the needs and frustrations of our user. Meet Kevin, our target user:

Design Phase

With a clear understanding of Kevin's requirements, we started with our design phase. Knowl's design was done in three steps:

a. Looking for inspiration

With so much information on the problem space and re-iteration of similar thoughts within ourselves, as a team, we realized that we were constrained in our vision of the solution. To get us to innovate solutions we tried out the worst possible idea method majorly concerning the feasibility.


It was a difficult task to perform given the sensitivity of the problem space, however navigating through that difficult space to find solutions led us to zoom out and think broader to generate solutions. We considered solutions ranging from AR glasses, and guidelines to digital applications. 

b. Low-fidelity prototypes

To make it possible for the solution to be easily integrated to our user's existing learning environment, we decided to come up with a minimal desktop application after considering a horde of other ideas ranging from AR glasses with captioning to a discreet projector for each student!


Our first sketches looked like these:

These sketches helped us come up with some low fidelity wireframes.

The next step was to conduct a cognitive walkthrough to work on each aspect of the application's functionality. In addition to our HMW statements, we also kept the following considerations in mind:

c. Mid-fidelity prototypes

We refined the low fidelity wireframes to build our mid-fidelity prototypes as shown below:

Next we conducted a quick concept evaluation and check-in with our instructor before moving to our final design

Final Design

After evaluating our mid-fi prototypes based on feedback received, we designed the first iteration of our final prototype.

The application is a floating widget, designed thus to minimize visual clutter in the student's learning environment.

  • With the "request" feature, Kevin, our user, can request for information with a single message with a timestamp.

  • Other Knowl users in the class can respond to his request in a timely manner to help him stay up-to-date with classroom information.

  • The "transcription" feature provides live transcription of the lecture. The main design consideration for this was a minimal distraction interface.

  • We also want to support the automatic labeling of peers and instructors in the transcript using the Voiceprint feature included in the Profile.

  • Switching to the "group mode" allows Kevin to use transcription for small group discussions.

  • Group members can be manually added or automatically detected via Bluetooth.

  • Kevin can also manually correct any incorrectly labeled team members in the transcript using the edit button.

  • In the Profile section, each Knowl user can add their details along with a VoicePrint.

  • Knowl will prompt the user to repeat some sentences to get an accurate picture of their voice tone, color, etc for effective voice recognition. This is used to generate labels for automatically generated transcripts.

Accessibility considerations:

  1. The size of the text was ensured to be readable on every screen size.

  2. We provided several Keyboard navigation options to improve usability.

  3. The floater could be accessed via a short cut that could be added to the accessibility center of the user's device.

  4. All the primary colors are accessible - The color contrast for primary color combinations (Icons, labels, buttons) complies with WCAG 2.2 AA 1.4.3.

  5. We also ensured that the report of our project met all guidelines for accessible PDF files.

Future scope

  1. Conduct detailed usability studies

  2. Integrate with mobile devices to encourage adoption and facilitate further ease of use .

  3. Integrate with LMS systems like Canvas for wider adoption.

Reflection

  • This project taught me the importance of striking a balance between constraints and exploration of possibilities, particularly in inclusive design.

  • We also learnt to be more conscious of our responsibilities as researchers particularly when involving vulnerable populations/

  • We also had an experience of how a thoughtfully designed product could sometimes be universal in adoption.