Choosing the best method for local validity estimation: Relative accuracy of meta-analysis versus a local study versus Bayes-Analysis

• 成果类型: Article
• 署名作者: Newman, Daniel A.; Jacobs, Rick R.; Bartram, Dave
• 署名单位: Texas A&M University System; Texas A&M University College Station; Pennsylvania Commonwealth System of Higher Education (PCSHE); Pennsylvania State University; Pennsylvania State University - University Park
• 刊物名称: JOURNAL OF APPLIED PSYCHOLOGY
• ISSN/ISSBN: 0021-9010
• DOI: 10.1037/0021-9010.92.5.1394
• 发表日期: 2007
• 页码: 1394-1413
• 关键词: validity generalization; Meta-analysis; ADVERSE IMPACT

摘要

This study assessed the relative accuracy of 3 techniques-local validity studies, meta-analysis, and Bayesian analysis-for estimating test validity, incremental validity, and adverse impact in the local selection context. Bayes-analysis involves combining a local study with nonlocal (meta-analytic) validity data. Using tests of cognitive ability and personality (conscientiousness) as predictors, an empirically driven selection scenario illustrates conditions in which each of the 3 estimation techniques performs best. General recommendations are offered for how to estimate local parameters, based on true population variability (sigma(2)(rho)) and the number of studies in the meta-analytic prior (k). Benefits of empirical Bayesian analysis for personnel selection are demonstrated, and equations are derived to help guide the choice of a local validity technique (i.e., meta-analysis vs. local study vs. Bayes-analysis).