Main Page
Replication Research of Social Sciences: Building Knowledge Together
Dai Li
Replication has been deemed important for scientific study. In physical sciences, it is common to have multiple teams replicating a piece of research to verify its validity. In recent years, social sciences have increasingly paid more attention to replication, in an effort to mimic the successful paradigm of phisical sciences.
However, replication research is less likely to be published. Generally speaking, a single piece of replication contributes little to the academic community. This has discouraged efforts in replicating and verifying pulished works. A new medium beyond journals is required to serve this purpose. This is the motivation behind this wiki site.
We define replication in the following orders:
- L0: Coding replication. Authors replicate the analysis from the replicated work with data and models unchanged, aiming to obtain identical results. This is to eliminate coding errors in published works, and to measure possible impacts of different versions of softwares or even their defects.
- L1: Sensitivity tests. Authors replicate the analysis from the replicated work on the same data, with models slightly tweaked in theoretically insignificant ways. This is to test the robustness of published works, and to find maneuvers such as cherry-picking, etc.
- L2: Model replication. Authors replicate the analysis from the replicated work on different sets of data, with models reasonably unchanged. For example, applying the same models on data from a different survey, a different nation, a different year, etc. This is to test the generalizability of the replicated work.
- L3: Theory replication. Authors replicate the analysis from the replicated work, changing data and/or models in an undrastic way that is potentially theoretically significant.
L2 and L3 are often solid components in articles published in low-tier journals. Their contributions are sometimes deemed too trivial to be published on top journals. This site encourages researchers to share their findings so as to build a more inclusive and comprehensive view.
