Indexing Dynamic Collective Constructs Using Computer-Aided Text Analysis: Construct Validity Evidence and Illustrations Featuring Team Processes

• 成果类型: Article
• 署名作者: Mathieu, John E.; Wolfson, Mikhail A.; Park, Semin; Luciano, Margaret M.; Bedwell-Torres, Wendy L.; Ramsay, P. Scott; Klock, Elizabeth A.; Tannenbaum, Scott, I
• 署名单位: University of Connecticut; University of Kentucky; University of Iowa; Pennsylvania Commonwealth System of Higher Education (PCSHE); Pennsylvania State University; Pennsylvania State University - University Park; University of Memphis; State University System of Florida; University of South Florida
• 刊物名称: JOURNAL OF APPLIED PSYCHOLOGY
• ISSN/ISSBN: 0021-9010
• DOI: 10.1037/apl0000856
• 发表日期: 2022
• 页码: 533-559
• 关键词: team process; CATA; construct validity; multitrait-multimethod

摘要

Organizational processes have been widely recognized as both multilevel and dynamic, yet traditional methods of measurements limit our ability to model and understand such phenomena. Featuring a popular model of team processes advanced by Marks et al. (2001), we illustrate a method to use individuals' communications as construct valid unobtrusive measures of collective constructs occurring over time. Thus, the purpose of this investigation is to develop computer-aided text analysis (CATA) techniques that can score members' communications into valid team process measures. We apply a deductive content validity-based method to construct CATA dictionaries for Marks et al.'s dimensions. We then demonstrate their convergent validity with subject matter experts' (SMEs) hand-coded team communications and different SMEs' behaviorally anchored rating scales based on video recordings of team interactions, using multitrait-multimethod analyses in two samples. Using a third sample of paramedics performing a high-fidelity mass casualty incident exercise, we further demonstrate the convergent validity of the CATA and SME scorings of communications. We then model the relationships among processes across episodes using all three samples. Next, we test criterion-related validity using a longitudinal dual-discontinuous change growth modeling design featuring the paramedic CATA-scored team processes as related to a dynamic performance criterion. Finally, we integrate behavioral data from wearable sensor badges to illustrate how CATA can be scored at the individual level and then leveraged to model dynamic networks of team interactions. Implications, limitations, directions for the future research, and guidelines for the application of these techniques to other collective constructs are discussed.