Skip to content

Quandt, Malzkuhn secure NSF grant for signing avatars research

August 6, 2021

Share:

The Office of Sponsored Programs and Research Services (OSPRS) congratulates Dr. Lorna Quandt and Ms. Melissa Malzkuhn, principal investigators, on their award from the National Science Foundation in the amount of $849,999. This award is for a project titled New Dimensions of ASL Learning: Implementing and Testing Signing Avatars and Immersive Learning (SAIL 2).

The highly-spatial three-dimensional nature of American Sign Language (ASL) has created a serious barrier to technology-supported ASL instruction. What if ASL learners could access high-quality ASL instruction from native sign language instructors through a virtual reality-based, game-like environment? This project launches from prior work on the NSF-funded Signing Avatars and Immersive Learning (SAIL) project. The SAIL project yielded a working prototype of an immersive sign language learning environment in virtual reality. The current project expands past the prototype stage into a fully-fledged ASL learning experience. 

In the new version of SAIL, called SAIL 2, the research team is developing a more complete system where users enter virtual reality and interact with signing avatars (computer-animated virtual humans built from motion capture recordings) who teach users ASL vocabulary. Access to signed language is key to healthy development for many deaf individuals, but it remains a major challenge when access to high-quality ASL instruction is limited by time and resources.

SAIL 2 sets a foundation for greater access to learning ASL, which has the potential for improving the lives of deaf children and adults. The project focuses on developing and testing this entirely novel ASL learning tool and fostering the inclusion of underrepresented minorities in STEM. This work has the potential to substantially advance the fields of virtual reality, ASL instruction, and embodied learning.

Immersive virtual reality is particularly well suited for highly spatial signed languages. The SAIL 2 project leverages head-mounted virtual reality and high-quality signing avatars to create a gamified ASL-learning system. SAIL 2 will be the only ASL learning system in virtual reality that does not require the user to wear specialized gloves or other peripheral devices. The project develops a functioning version of the comprehensive SAIL 2 system, and user testing during the design process guides the details of the development. Key features of the system include sign recognition through hand tracking cameras, corrective feedback, and a gamified experience. Following the design and development of SAIL 2, the research team conducts behavioral research to evaluate the learning outcomes of SAIL 2. Evaluation of specific learning outcomes includes both the understanding of ASL vocabulary and the accuracy of sign production. Because of the embodied nature of signed language, mechanistic measures of the neural substrates of learning, including engagement of the sensorimotor cortices, are obtained through electroencephalography (EEG). The patterns of neural oscillatory activity provide insight into short-term changes in brain activity associated with using SAIL 2. The cognitive neuroscience experiment builds on previous research identifying the neural processes supporting sign language perception, and overall this project extends technological advances in high-fidelity motion capture recordings, avatar creation, and virtual reality.

Details of the award are as follows:

  • Dr. Lorna Quandt and Ms. Melissa Malzkuhn Principal Investigators
  • Ph.D, in Educational Neuroscience
  • National Science Foundation
  • Award No.: 2118742
  • Dollar amount of federal funds awarded: $849,999 from 10/01/2021 to 09/30/2024
  • Percentage total cost of the program financed with federal funding: 100%
  • Percentage and dollar amount of the total costs funded by non-governmental sources: 0%

Congratulations to Dr. Quandt and Ms. Malzkuhn!

6 August 2021

Share:

About the Author

Gallaudet's Bison mascot pointing at you.
Chapel Hall:102E
202-250-2411 VideoPhone

Recent Posts

Roberto E. Wirth, E-’74 & H-’09, passed away on June 5 in Rome, Italy. Mr. Wirth was owner and managing director of the Hotel Hassler in Rome, one of the most prestigious family-owned hotels in the world, and owned several other hotels and resorts throughout Italy. He was a strong advocate for deaf people in...

Alumnus Timel Benton has signed a contract with the Bay Area Panthers of the Indoor Football League (IFL). Benton, who graduated last month, is the first Gallaudet Bison to sign a professional football contract since Tony Tatum signed on with the Utah Blaze in the now-disbanded Arena Football League (AFL) in May 2013. Benton was...

James Caverly, ’11, who plays Theo Dimas in Hulu’s Only Murders in the Building, will play Professor Harold Hill in the Olney Theatre Center’s summer production of Meredith Willson’s Tony-winning musical The Music Man, which opens tonight and runs through July 23. The show’s official opening is on Thursday, June 23. Sandra Mae Frank, ’13,...

About the Author

Robert Weinstock

Published Articles: 110

Recent Posts
Deaf hotelier Roberto Wirth passes away
Alumnus Timel Benton to play professional football
Alumni, faculty featured in The Music Man at Olney Theatre Center
Related Categories
Media Inquiries

For any other media inquiry, please contact:

No media contact found!

Stay up to date on all the Gallaudet happenings, both stories and initiatives we are doing with our Signing community!​

Admissions Requirements

Hearing Undergraduate