Mapping Methodological Variation in Experience-Sampling Research From Design to Data Analysis: A Systematic Review

Abstract

The experience-sampling method (ESM) has become a widespread tool to study time-varying constructs across many subfields of psychological and psychiatric research. Variety in subfields of research and constructs of interest has contributed to considerable methodological variation. Therefore, in this systematic review, we aimed to (a) describe the methodological variation in ESM-study designs and (b) assess the transparency (i.e., reporting and open-science practices) of ESM studies. We developed an extensive list of data-extraction items covering the entire workflow of an ESM study, from conception of the research question to reporting the results. These data were extracted from 150 recently published articles describing 162 studies applying ESM in the field of psychology and psychiatry. As expected, variation was observed for every methodological decision. Some findings aligned with previous syntheses of the literature (e.g., a median study duration of 10 days, a majority of studies including convenience samples of the general population, the high prevalence of multilevel modeling). Regarding other decisions, this review was able to provide new insights (e.g., questionnaire length and content often varied between and/or within participants, average sample sizes have increased over time). Transparency in reporting and open-science practices seem to have improved in recent years, but there is room for further development of awareness and guidance. This broad overview of ESM research serves as a first step toward a broader goal to improve the methodological quality of ESM research in psychology, contributing to a more rigorous, credible science of daily life.