Examining the generalizability of research findings from archival data

Andrew Delios; Elena Giulia Clemente; Tao Wu; Hongbin Tan; Yong Wang; Michael Gordon; Domenico Viganola; Zhaowei Chen; Anna Dreber; Magnus Johannesson; Thomas Pfeiffer; Generalizability Tests Forecasting Collaboration; Jais Adam-Troian; Eric Luis Uhlmann

doi:10.1073/pnas.2120377119

Examining the generalizability of research findings from archival data

Andrew Delios^*, Elena Giulia Clemente, Tao Wu, Hongbin Tan, Yong Wang, Michael Gordon, Domenico Viganola, Zhaowei Chen, Anna Dreber, Magnus Johannesson, Thomas Pfeiffer, Generalizability Tests Forecasting Collaboration, Jais Adam-Troian, Eric Luis Uhlmann^*

^*Corresponding author for this work

School of Social Sciences

Research output: Contribution to journal › Article › peer-review

33 Citations (Scopus)

14 Downloads (Pure)

Abstract

The extent to which results from complex datasets generalize across contexts is critically important to numerous scientific fields as well as to practitioners who rely on such analyses to guide important strategic decisions. Our initiative systematically investigated whether findings from the field of strategic management would emerge in new time periods and new geographies. Original findings that were statistically reliable in the first place were typically obtained again in novel tests, suggesting surprisingly little sensitivity to context. For some social scientific areas of inquiry, results from a specific time and place can be a meaningful guide as to what will be observed more generally.

Original language	English
Article number	e2120377119
Journal	Proceedings of the National Academy of Sciences of the United States of America
Volume	119
Issue number	30
DOIs	https://doi.org/10.1073/pnas.2120377119
Publication status	Published - 19 Jul 2022

Keywords

Research reliability
generalizability
archival data
reproducibility
context sensitivity

Access to Document

10.1073/pnas.2120377119Licence: CC BY-NC-ND

Download pdfFinal published version, 843 KBLicence: CC BY-NC-ND

Cite this

Delios, A., Clemente, E. G., Wu, T., Tan, H., Wang, Y., Gordon, M., Viganola, D., Chen, Z., Dreber, A., Johannesson, M., Pfeiffer, T., Generalizability Tests Forecasting Collaboration, Adam-Troian, J., & Uhlmann, E. L. (2022). Examining the generalizability of research findings from archival data. Proceedings of the National Academy of Sciences of the United States of America, 119(30), Article e2120377119. https://doi.org/10.1073/pnas.2120377119

@article{127a404429c2467d9bf960dd9c2e5213,

title = "Examining the generalizability of research findings from archival data",

abstract = "The extent to which results from complex datasets generalize across contexts is critically important to numerous scientific fields as well as to practitioners who rely on such analyses to guide important strategic decisions. Our initiative systematically investigated whether findings from the field of strategic management would emerge in new time periods and new geographies. Original findings that were statistically reliable in the first place were typically obtained again in novel tests, suggesting surprisingly little sensitivity to context. For some social scientific areas of inquiry, results from a specific time and place can be a meaningful guide as to what will be observed more generally.",

keywords = "Research reliability, generalizability, archival data, reproducibility, context sensitivity",

author = "Andrew Delios and Clemente, \{Elena Giulia\} and Tao Wu and Hongbin Tan and Yong Wang and Michael Gordon and Domenico Viganola and Zhaowei Chen and Anna Dreber and Magnus Johannesson and Thomas Pfeiffer and \{Generalizability Tests Forecasting Collaboration\} and Jais Adam-Troian and Uhlmann, \{Eric Luis\}",

year = "2022",

month = jul,

day = "19",

doi = "10.1073/pnas.2120377119",

language = "English",

volume = "119",

journal = "Proceedings of the National Academy of Sciences of the United States of America",

issn = "0027-8424",

publisher = "National Academy of Sciences",

number = "30",

}

Delios, A, Clemente, EG, Wu, T, Tan, H, Wang, Y, Gordon, M, Viganola, D, Chen, Z, Dreber, A, Johannesson, M, Pfeiffer, T, Generalizability Tests Forecasting Collaboration, Adam-Troian, J & Uhlmann, EL 2022, 'Examining the generalizability of research findings from archival data', Proceedings of the National Academy of Sciences of the United States of America, vol. 119, no. 30, e2120377119. https://doi.org/10.1073/pnas.2120377119

TY - JOUR

T1 - Examining the generalizability of research findings from archival data

AU - Delios, Andrew

AU - Clemente, Elena Giulia

AU - Wu, Tao

AU - Tan, Hongbin

AU - Wang, Yong

AU - Gordon, Michael

AU - Viganola, Domenico

AU - Chen, Zhaowei

AU - Dreber, Anna

AU - Johannesson, Magnus

AU - Pfeiffer, Thomas

AU - Generalizability Tests Forecasting Collaboration

AU - Adam-Troian, Jais

AU - Uhlmann, Eric Luis

PY - 2022/7/19

Y1 - 2022/7/19

N2 - The extent to which results from complex datasets generalize across contexts is critically important to numerous scientific fields as well as to practitioners who rely on such analyses to guide important strategic decisions. Our initiative systematically investigated whether findings from the field of strategic management would emerge in new time periods and new geographies. Original findings that were statistically reliable in the first place were typically obtained again in novel tests, suggesting surprisingly little sensitivity to context. For some social scientific areas of inquiry, results from a specific time and place can be a meaningful guide as to what will be observed more generally.

AB - The extent to which results from complex datasets generalize across contexts is critically important to numerous scientific fields as well as to practitioners who rely on such analyses to guide important strategic decisions. Our initiative systematically investigated whether findings from the field of strategic management would emerge in new time periods and new geographies. Original findings that were statistically reliable in the first place were typically obtained again in novel tests, suggesting surprisingly little sensitivity to context. For some social scientific areas of inquiry, results from a specific time and place can be a meaningful guide as to what will be observed more generally.

KW - Research reliability

KW - generalizability

KW - archival data

KW - reproducibility

KW - context sensitivity

U2 - 10.1073/pnas.2120377119

DO - 10.1073/pnas.2120377119

M3 - Article

SN - 0027-8424

VL - 119

JO - Proceedings of the National Academy of Sciences of the United States of America

JF - Proceedings of the National Academy of Sciences of the United States of America

IS - 30

M1 - e2120377119

ER -

Examining the generalizability of research findings from archival data

Abstract

Keywords

Access to Document

Fingerprint

Cite this