EDUCATION
By Shannon Magsam | Photos by Sean Rhomberg
U of A Leads Multi-University Project to Develop New AI-Enhanced Technologies
The University of Arkansas is developing new AI-enhanced technologies for people with speech and language limitations in collaboration with Florida State, Auburn, Penn State and Temple universities.
​
The goal is to improve augmentative and alternative communication devices, or AAC devices, so people with developmental disabilities can communicate more effectively and precisely with the world around them. Tools that foster communication and language can help individuals achieve greater success in various aspects of life, including work, school, friendships and community engagement.
​
A faculty team from the U of A College of Education and Health Professions is leading the $4.8 million project. Dr. Christine Holyfield, an associate professor of communication sciences and disorders and the principal investigator on the five-year Department of Health and Human Services grant, is joined by two of the college’s special education faculty members, Drs. Elizabeth Lorah and Sheida Raley.
Dr. Elizabeth Lorah strategizes with doctoral student Madison Maddox in the AAC research lab
AAC technology allows users to select words or pictures on a screen to generate corresponding speech. However, there’s much room for improvement. This research is deeply important for AAC users, who often face significant challenges in engaging in conversations. Their communication can take time, and it’s frustrating when discussions move forward before they have the chance to share their thoughts, insights or even jokes.
​
The interdisciplinary team of communication disorder scientists, special educators and engineers from the various universities is working closely with people who use AAC devices to ensure the enhancements are meaningful for them.
​
The research is already underway, with faculty members working with children in various settings. For example, Lorah is exploring how children can program their own AAC devices by taking photos of their environment, allowing AI to ascribe related words using computer vision. For example, a child with Down syndrome could spot an interesting insect on a class nature walk, and by taking a photo with her AAC device, she could tell a friend about the cool thing she saw. “It’s empowering for children to choose words and photo representations for themselves, rather than learning the words and images that others choose for them,” Lorah said.
Holyfield is exploring ways that location can inform prediction options for communication. This spring, she’s starting a new study that involves collaborating with classrooms and clinics that have dedicated zones, such as reading areas, dramatic play spaces or art corners. The project’s goal is to enhance in-the-moment communication options for children based on the specific zone they’re visiting. For instance, if a child enters the reading area, the communication technology would predict related options for the child to communicate with classmates or professionals, such as “Want to read together?”
​
Currently, AAC users must invest a lot of time learning how to operate their devices before they can use them effectively.
College of Education and Health Professions faculty members Sheida Raley, Christine Holyfield and Elizabeth Lorah
Doctoral student Madison Maddox works with a child using an AAC device
As AI begins to “learn” more about individual users, conversations can become faster-paced and feel more personal. For example, technology might detect an AAC user’s gestures or facial expressions and modulate speech output according to their communicative intent, which would likely spark a more authentic response from the person they’re communicating with, a key component of language development.
This grant builds on the work of Holyfield and Lorah in the U of A College of Education and Health Professions AAC Lab, which advances research in Arkansas. Multiple WE CARE grants from the college have supported the lab. WE CARE, which stands for Wellness and Education Commitment to Arkansas Excellence, is the college’s core mission.