A world-class institute of changemakers in the deaf and signing community.
Since 1864, we have been investing in and creating resources for deaf and hard of hearing children, their families, and the professionals who work with them.
Over 50 degree programs, with online and continuing education for personal and professional development.
Innovating solutions to break down barriers, and using science to prove what does and doesn’t work.
We make it easy for you to apply and enter here.
Ready to take the next step toward a college education?
Make lasting memories and grow in ways you never thought possible.
Merrill Learning Center (MLC), Upper Level, Room 2200
Gallaudet commissioned research studies on the ASLPI to:
Importance of Research:
Defining Validity of an assessment: Validity refers to the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests. ( Standards, pg. 9)
Defining Reliability of an assessment: Reliability refers to the consistency of measurements when the testing procedure is repeated on a population of individuals or groups. ( Standards, pg. 25)
ASLPI is rater-based assessment. Therefore, the ASLPI interviewers must be skilled and strongly focused on the construct being measured throughout the interview process.
Measurement Structure of the ASL Proficiency Construct as Assessed by the ASLPI
The findings provided evidence that a majority of the variability in the five dimensions was attributable to a single higher-order construct. That is, a majority of the variability in dimension ratings can be meaningfully traced to a single underlying factor, which is as intended by the test developer. Moreover, the relationships between the dimensions and the single underlying factor were strongly consistent (i.e., approximately equal in magnitude). In sum, the five dimensions are representative of a single underlying construct, and the dimensions relate to the underlying construct in a consistent manner, these findings align with the design and current rating protocol of the ASLPI.
Preliminary Validity Study Conclusion:
Study Sample: 1568 assessment interviews rated by ASLPI evaluators
Inter and intra-rater Reliability - measure of consistency between ratings of the same test taker from different evaluators. This is an "intra-class correlation" (ICC) for a single rater or the average rating across 3 raters.
Correlation Across All Possible Evaluator Pairs
Repeatability of ASLPI Ratings
SWA Consulting, Inc (SWA)311 S. Harrington St., Suit 200Raleigh, NC 27603Dr. Eric A. Surface is the president of SWA Consulting, Inc. (SWA; formerly Surface, Ward, & Associates), a management consulting and applied personnel research firm based in Raleigh, NC. SWA focuses on developing evidence-based human performance solutions for clients. For over a decade, Dr. Surface has worked with military, non-profit, and private-sector organizations-such as the United States Special Operations Command, the American Council on the Teaching of Foreign Languages, and IBM-on projects related to performance, training, work-related foreign language, work analysis, testing, and organizational effectiveness. Dr. Surface has been published in Personnel Psychology, Organizational Research Methods, Human Performance, Military Psychology, Journal of Management, and Foreign Language Annals. He has presented numerous papers at academic, professional, and military conferences and has served as a reviewer for several conferences and journals. His research interests include training effectiveness (broadly defined), the interaction of context and individual differences to influence criteria, foreign language proficiency testing and training, the use of technology for training delivery, test validation, and survey methodology. Dr. Surface earned his PhD in industrial/organizational psychology from NCSU, his MA from East Carolina University, and his BA from Wake Forest University. He is a former Consortium Research Fellow and Consortium Post-Doctoral Research Fellow with the Army Research Institute.
In 2010-2012, the ASLPI evaluation system underwent validity and reliability studies by an external research and consulting firm. The lead researcher traveled to Gallaudet University to present about the findings to the Gallaudet faculty and administration (outcomes posted above).Upon conclusion of this two year research study, the following recommendations were prioritized by ASL Diagnostic and Evaluation Services:
During the summer of 2013 from June through August, The ASLPI conducted a comprehensive and careful analysis of the Functional Descriptions for the 0-5 proficiency levels, including the plus (+) levels. After 500+ hours and critical analysis of 90+ video recorded ASLPI evaluations, updated and enhanced Functional Descriptions were finalized.
At the beginning of fall 2013, all ASLPI Evaluators were required to participate in a mandatory training. Adherence to stricter standards for rating agreement among evaluators was also instituted. The absolute agreement approach for rating decisions was implemented as is recommended for high stakes testing.
Following the training and implementation of the stricter rating expectation (absolute agreement for proficiency level decisions), The ASLPI established a monitoring system. Prior to distribution, every rating decision went through a review process to examine inter- and intra-rater reliability. This ensured application and effectiveness of the training. At the end of fall 2013, The ASLPI conducted a rating reliability study across all evaluators. The reliability study included the 275 evaluations conducted. Rating reliability for the entire pool of ASLPI Evaluators was determined to be 84%. For high stakes language proficiency testing, reliability must be 80% or above.
The ASLPI is continuing to closely monitor all evaluators to ensure that a shared mental model is consistent and evaluators are applying the same operational definitions of the proficiency constructs.
Gallaudet University is a federally chartered private and premier university for the deaf and hard of hearing since 1864.
Copyright © 2021 Gallaudet University. All rights reserved.
800 Florida Avenue NE Washington, D.C. 20002
Spring 2021 – Dec 12Fall 2021 – May 15