Skip to content

Quandt, Malzkuhn secure NSF grant for signing avatars research

August 6, 2021

Share:

The Office of Sponsored Programs and Research Services (OSPRS) congratulates Dr. Lorna Quandt and Ms. Melissa Malzkuhn, principal investigators, on their award from the National Science Foundation in the amount of $849,999. This award is for a project titled New Dimensions of ASL Learning: Implementing and Testing Signing Avatars and Immersive Learning (SAIL 2).

The highly-spatial three-dimensional nature of American Sign Language (ASL) has created a serious barrier to technology-supported ASL instruction. What if ASL learners could access high-quality ASL instruction from native sign language instructors through a virtual reality-based, game-like environment? This project launches from prior work on the NSF-funded Signing Avatars and Immersive Learning (SAIL) project. The SAIL project yielded a working prototype of an immersive sign language learning environment in virtual reality. The current project expands past the prototype stage into a fully-fledged ASL learning experience. 

In the new version of SAIL, called SAIL 2, the research team is developing a more complete system where users enter virtual reality and interact with signing avatars (computer-animated virtual humans built from motion capture recordings) who teach users ASL vocabulary. Access to signed language is key to healthy development for many deaf individuals, but it remains a major challenge when access to high-quality ASL instruction is limited by time and resources.

SAIL 2 sets a foundation for greater access to learning ASL, which has the potential for improving the lives of deaf children and adults. The project focuses on developing and testing this entirely novel ASL learning tool and fostering the inclusion of underrepresented minorities in STEM. This work has the potential to substantially advance the fields of virtual reality, ASL instruction, and embodied learning.

Immersive virtual reality is particularly well suited for highly spatial signed languages. The SAIL 2 project leverages head-mounted virtual reality and high-quality signing avatars to create a gamified ASL-learning system. SAIL 2 will be the only ASL learning system in virtual reality that does not require the user to wear specialized gloves or other peripheral devices. The project develops a functioning version of the comprehensive SAIL 2 system, and user testing during the design process guides the details of the development. Key features of the system include sign recognition through hand tracking cameras, corrective feedback, and a gamified experience. Following the design and development of SAIL 2, the research team conducts behavioral research to evaluate the learning outcomes of SAIL 2. Evaluation of specific learning outcomes includes both the understanding of ASL vocabulary and the accuracy of sign production. Because of the embodied nature of signed language, mechanistic measures of the neural substrates of learning, including engagement of the sensorimotor cortices, are obtained through electroencephalography (EEG). The patterns of neural oscillatory activity provide insight into short-term changes in brain activity associated with using SAIL 2. The cognitive neuroscience experiment builds on previous research identifying the neural processes supporting sign language perception, and overall this project extends technological advances in high-fidelity motion capture recordings, avatar creation, and virtual reality.

Details of the award are as follows:

  • Dr. Lorna Quandt and Ms. Melissa Malzkuhn Principal Investigators
  • Ph.D, in Educational Neuroscience
  • National Science Foundation
  • Award No.: 2118742
  • Dollar amount of federal funds awarded: $849,999 from 10/01/2021 to 09/30/2024
  • Percentage total cost of the program financed with federal funding: 100%
  • Percentage and dollar amount of the total costs funded by non-governmental sources: 0%

Congratulations to Dr. Quandt and Ms. Malzkuhn!

6 August 2021

Share:

About the Author

Gallaudet's Bison mascot pointing at you.
Chapel Hall:102E
202-250-2411 VideoPhone

Recent Posts

The Graduate Student Resource Center (GSRC) has recently launched. GSRC is a one-stop center for graduate students to access the resources they need to succeed in their programs, as mentioned in President Cordano’s Welcome Home address. The Center’s offerings include academic support services, career and professional development, well-being resources, and much more!  The Center can...

Gallaudet’s presence on the global stage requires a strong marketing team that not only supports growing enrollment, but also elevates the Gallaudet brand and mission. To achieve this, Brandi Rarus, Chief Marketing and Undergraduate Admissions Officer, has expanded and realigned the departments within University Communications and Undergraduate Admissions. Recognizing this growth, Elise Nye, ’94, has...

Gallaudet Technology Services (GTS) is changing how it distributes technology tools to its community at Gallaudet University and the Clerc Center. Each community member will receive a device, such as an iPad, laptop, or smartphone (determined as the best fit for their roles on campus), that they can use for their academic studies or work-related...

Related Events

8:00 am
Gallaudet University
[tribe_event_inline id="5639"]{venue:name}[/tribe_event_inline]
12:00 am
Gallaudet University
[tribe_event_inline id="5643"]{venue:name}[/tribe_event_inline]
8:00 am
Gallaudet University
[tribe_event_inline id="14294"]{venue:name}[/tribe_event_inline]
Chapel Hall
200
About the Author

Robert Weinstock

Published Articles: 50

Recent Posts
Announcing the Graduate Student Resource Center
University Communications and Undergraduate Admissions: Working Together to Elevate Gallaudet University
Connected Gallaudet Update
Related Categories
Media Inquiries

For any other media inquiry, please contact:

No media contact found!

Stay up to date on all the Gallaudet happenings, both stories and initiatives we are doing with our Signing community!​

Admissions Requirements

Hearing Undergraduate