Initial Investigation Into Computer Scoring of Candidate Essays for Personnel Selection

• 成果类型: Article
• 署名作者: Campion, Michael C.; Campion, Michael A.; Campion, Emily D.; Reider, Matthew H.
• 署名单位: University of South Carolina System; University of South Carolina Columbia; Purdue University System; Purdue University; State University of New York (SUNY) System; University at Buffalo, SUNY
• 刊物名称: JOURNAL OF APPLIED PSYCHOLOGY
• ISSN/ISSBN: 0021-9010
• DOI: 10.1037/apl0000108
• 发表日期: 2016
• 页码: 958-975
• 关键词: adverse impact; big data; personnel selection; statistics; test scoring

摘要

Emerging advancements including the exponentially growing availability of computer-collected data and increasingly sophisticated statistical software have led to a Big Data Movement wherein organizations have begun attempting to use large-scale data analysis to improve their effectiveness. Yet, little is known regarding how organizations can leverage these advancements to develop more effective personnel selection procedures, especially when the data are unstructured (text-based). Drawing on literature on natural language processing, we critically examine the possibility of leveraging advances in text mining and predictive modeling computer software programs as a surrogate for human raters in a selection context. We explain how to train a computer program to emulate a human rater when scoring accomplishment records. We then examine the reliability of the computer's scores, provide preliminary evidence of their construct validity, demonstrate that this practice does not produce scores that disadvantage minority groups, illustrate the positive financial impact of adopting this practice in an organization (N similar to 46,000 candidates), and discuss implementation issues. Finally, we discuss the potential implications of using computer scoring to address the adverse impact-validity dilemma. We suggest that it may provide a cost-effective means of using predictors that have comparable validity but have previously been too expensive for large-scale screening.