Revisiting Sackett et al.'s (2022) Rationale Behind Their Recommendation Against Correcting for Range Restriction in Concurrent Validation Studies

• 成果类型: Editorial Material
• 署名作者: Oh, In-Sue; Le, Huy; Roth, Philip L.
• 署名单位: Pennsylvania Commonwealth System of Higher Education (PCSHE); Temple University; University of Texas System; University of Texas at San Antonio; Clemson University; Pennsylvania Commonwealth System of Higher Education (PCSHE); Temple University
• 刊物名称: JOURNAL OF APPLIED PSYCHOLOGY
• ISSN/ISSBN: 0021-9010
• DOI: 10.1037/apl0001078
• 发表日期: 2023
• 页码: 1300-1310
• 关键词: range restriction; Meta-analysis; personnel selection; validation; concurrent validity

摘要

Sackett et al. (2022) recommend against correcting for range restriction (RR) in concurrent validation studies. The main rationale behind their recommendation is that unless r(zx) (an unrestricted true-score correlation between the third variable Z where actual selection occurred in a top-down manner [a.k.a., suitability] and the predictor of interest, X) is as high as .90 and selection ratios are as low as .30-both unlikely events in their view, the degree of RR (u(x)) in concurrent validation studies is unlikely to be low enough (i.e., lower than .90) to warrant RR correction. That is, (a) the r(zx) = .90 and (b) the selection ratio =.30 are two critical conditions for the third condition, (c) u(x) = .90, a need for RR correction. In this study, we revisit each of these conditions that constitute the rationale behind their recommendation: (a) whether r(zx) is unlikely to be as high as .90; (b) whether selection ratios of .30 or lower are extreme; and (c) whether the degree of RR is little to no (i.e., u(x) = .90) in concurrent validation studies, thus no need for correcting for RR in concurrent validation studies. First, our reanalysis of their Table 1 indicates that it is not implausible that r(zx) is as high as .90. Second, several studies report that selection ratios of .30 or lower are not extreme. Finally, our reanalysis of their Table 5 indicates that Sackett et al. substantially underestimate the severity of RR and its biasing effect on operational validity in concurrent validation studies due to their use of a particular RR correction method (Case IV). We believe these findings suggest that there is not sufficient support for the rationale behind Sackett et al.'s recommendation and, thus, their recommendation itself should be reconsidered.