Main Page

Replication Research of Social Sciences: Building Knowledge Together

Motivation

Replication has been deemed important for scientific study. In physical sciences, it is common to have multiple teams replicating a piece of research to verify its validity. In recent years, social sciences have drawn increasingly attention to replication, in an effort to mimic the successful paradigm of phisical sciences.

However, replication research is less likely to be published. Generally speaking, a single piece of replication contributes little to the academic community. This has discouraged efforts in replicating and verifying pulished works. A new medium beyond journals is required to serve this purpose. This is the motivation behind this wiki site.

Contents

Following Freese and Peterson (2017) <ref>Jeremy Freese and David Peterson, 2017, Replication in Social Science, Annual Review of Sociology 43:1, 147-165.</ref>, we define replication in the following orders:

• L0: Verifiability.
  • Same data, same models. Authors replicate the analysis from the replicated work with data and models unchanged, aiming to obtain identical results. This is to eliminate coding errors in published works, and to measure possible impacts of different versions of softwares or even their defects. Therefore, L0 pages are strongly encouraged to report the name and version of software and/or packages, and other technical details to facilitate replication.
• L1: Robustness.
  • Same data, different models. Using the same method with different specifications or using a different method aiming for the same results both fall into this category. Authors replicate the analysis from the replicated work on the same data, with models slightly tweaked in theoretically insignificant ways. This is to test the robustness of published works, and to find maneuvers such as cherry-picking, p hacking, etc.
• L2: Repeatablility.
  • Different data, same models. The models should be reasonably unchanged to test if the same results can be observed in a different sample. Authors replicate the analysis from the replicated work on different sets of data, with models reasonably unchanged. For example, applying the same models on data from a different survey, a different nation, a different year, etc.
• L3: Generalizability.
  • Different data, different models. Authors replicate the analysis from the replicated work, changing data and/or models in an undrastic way that is potentially theoretically significant.

By deconstructing published works, verifying their analytical components and building upon each other, authors across the globe can take advantage of new tools to weave a graph of social sciences findings. The results are digitized and structured, easily processed and quantified by algorithms. We all will benefit from this all-encompassing graph.

We especially encourage graduate students to participate in this endeavor. Replication is a great way of learning, and is already part of the curriculum in many graduate programs. Their hard work, no matter how trivial in appearance, should not go to waste but contribute to the community's knowledge.

Demo

The following are strictly for demonstration purpose. Please do not distribute these papers without permission):

Replicated:

• The Tragedy of the Nomenklatura: Career Incentives and Political Radicalism during China's Great Leap Famine, Kung & Chen, 2011. Source

Replication:

• A Tragedy of the Nomenklatura? Career incentives, political loyalty and political radicalism during China's Great Leap Forward, Yang, Xu & Tao, 2014. Source

Getting started

• Editing Guidelines

Notes

<references />