Improving the Measurement of Group-Level Constructs by Optimizing Between-Group Differentiation

• 成果类型: Article
• 署名作者: Bliese, Paul D.; Maltarich, Mark A.; Hendricks, Jonathan L.; Hofmann, David A.; Adler, Amy B.
• 署名单位: University of South Carolina System; University of South Carolina Columbia; University of North Carolina; University of North Carolina Chapel Hill; United States Department of Defense; United States Army; Walter Reed Army Institute of Research (WRAIR)
• 刊物名称: JOURNAL OF APPLIED PSYCHOLOGY
• ISSN/ISSBN: 0021-9010
• DOI: 10.1037/apl0000349
• 发表日期: 2019
• 页码: 293-302
• 关键词: multilevel; reliability; validity; measurement

摘要

The ability to detect differences between groups partially impacts how useful a group-level variable will be for subsequent analyses. Direct consensus and referent-shift consensus group-level constructs are often measured by aggregating group member responses to multi-item scales. We show that current measurement validation practice for these group-level constructs may not be optimized with respect to differentiating groups. More specifically, a 10-year review of multilevel articles in top journals reveals that multilevel measurement validation primarily relies on procedures designed for individual-level constructs. These procedures likely miss important information about how well each specific scale item differentiates between groups. We propose that group-level measurement validation be augmented with information about each scale item's ability to differentiate groups. Using previously published datasets, we demonstrate how ICC(1) estimates for each item of a scale provide unique information and can produce group-level scales with higher ICC(1) values that enhance predictive validity. We recommend that researchers supplement conventional measurement validation information with information item-level ICC(1) values when developing or modifying scales to assess group-level constructs.