Myth-busting technology gives people with non-verbal autism a voice

There is no shortage of myths when it comes to autism.

From it’s a disease, to vaccines are the cause, to autistic people don’t feel emotion, many challenges remain in getting people to understand autism.

Another myth, that autistic people have an intellectual disability and can’t speak, became the focus of a team of fourth-year Schulich School of Engineering students for their year-end capstone project.

Using a mixed-reality headset, the Microsoft HoloLens 2, the team was able to open the eyes of many who stopped at their booth during the annual Engineering Design Fair on April 5.

“The beauty of our project is that it allows us not to focus on what an individual can’t do, but design an application that builds on the things a person with non-verbal autism can do,” says project lead Andrey Balasanyan.

Interacting with holograms

When deciding on what project to choose, the team — comprising Balasanyan; Kyle DeBoer; Travis Dow, BSc’18; Libby Bell; Hunter Mooney; Jash Dubal; and Shabbir Hussain — wanted to make an impact on people.

Adding in some experience in mentoring students and backgrounds in virtual reality applications, this project spoke to them right away.

The team set a goal of gradually improving the motor function of autistic individuals by increasing their ability to interact with the holograms.

“The motor-function difficulty is designed to progressively increase in difficulty to challenge the user to improve their motor skills,” Balasanyan says. “For example, allowing the buttons to become smaller and smaller throughout the lesson plan and incorporating various interactive features.”

The team worked on the software under the watchful eyes of sponsor and UCalgary professor Dr. Diwakar Krishnamurthy, PhD, as well as project adviser and fellow professor Dr. Adam McCrimmon, BA’00, PhD’10.

Inspiring change

Krishnamurthy first became interested in the topic while reading two memoirs written by non-speaking autistic men: Ido in Autism Land: Climbing Out of Autism’s Silent Prison by Ido Kedar and The Reason I Jump: The Inner Voice of a Thirteen-Year-Old Boy with Autism by Naoko Higashida.

He says about 30 per cent of people on the spectrum can’t reliably communicate using speech.

Non-speakers are often misunderstood with many professionals and educators falsely equating not speaking with not thinking. These two books paint a moving picture of the devastating consequences of such an assumption.

A professor in Schulich’s Department of Electrical and Software Engineering, Krishnamurthy was inspired to use his expertise to find a technological solution.

He spends his research time focused on developing customized, game-based augmented reality (AR) apps where non-speakers can learn motor skills situated within meaningful education content.

“I feel such technology can be a powerful tool for non-speaking autistic people to access social and educational opportunities not typically offered to them,” Krishnamurthy says.

Speaking for the future

The team members were excited to see positive reaction to their project from not only their advisers, but also from those who put on the technology to see the world through a new set of eyes.

After the fair, they took some time to work out any last-minute bugs before handing the code off to the UCalgary AR/MR research group, co-led by Krishnamurthy.

“It’s exciting because this project doesn’t end here as things can always be improved or refined, and new innovation techniques can always be used,” Balasanyan says. “This application has a lot of potential and could one day offer autistic people some social and economic opportunities that might be out of reach today.”

Krishnamurthy was impressed by the team’s effort and level of engagement to make sure they had a product that could have the biggest impact.

“The prototype they built embodies many of the things the non-speaking community has been asking for,” he says. “Their work has laid the foundation for many interesting future research projects and development activities in my group.”

Krishnamurthy adds the product is a proof of concept, and that a version for preliminary user testing could be ready by the end of the year.