✦ScienexySCIENCE KNOWLEDGE LIBRARY
psychology-learning · SCI-PSY-001-028

Replication and Reproducibility in Psychology

Replication and reproducibility are practices that strengthen the credibility of psychological research. Replication examines whether findings can be obtained again in new investigations, while reproducibility concerns whether the same results can be obtained using the same data, methods and analytical procedures.

Explanation

A fuller explanation is being developed. Verify this concept with cited sources before relying on it.

Example or application

A worked example has not yet been added.

Conceptual Coverage

Definition

Replication

Replication is a new investigation designed to examine whether a previously reported finding can be observed again.

Reproducibility

Reproducibility generally refers to obtaining consistent computational results using the same data, code, methods and analytical procedures.

Terminology Note

The terms replication and reproducibility are sometimes used differently across scientific disciplines. Their precise meanings should be stated in each research context.

Types

NameDescription

Direct replication

Attempts to repeat an earlier study's procedures as closely as practical.

Conceptual replication

Tests the same theoretical idea using different methods, measurements or operational definitions.

Systematic replication

Repeats a study while deliberately examining additional populations, contexts or conditions.

Computational reproducibility

Checks whether reported analytical results can be regenerated from the original data and code.

Independent replication

Tests a finding through research conducted independently of the original research team.

Importance

  • Tests the stability of psychological findings.

  • Identifies findings that depend on specific samples or procedures.

  • Improves confidence in scientific conclusions.

  • Reveals methodological weaknesses.

  • Supports cumulative scientific knowledge.

Factors Affecting Replication

  • Sample characteristics

  • Measurement quality

  • Research context

  • Study design

  • Statistical power

  • Publication and reporting practices

  • Unreported methodological decisions

Scientific Foundation

Historical Development

Replication has long been a fundamental scientific practice. In psychology, concerns about inconsistent findings and research practices led to increased attention to preregistration, open data, transparent methods, multi-lab studies and independent replication.

Important Contributors

NameContribution

Brian Nosek

Contributed to open science practices and initiatives designed to improve research transparency.

John Ioannidis

Influenced scientific discussion about the reliability of published research findings.

Open Science Collaboration

Conducted a major collaborative project examining the reproducibility of published psychological research.

Major Research Initiatives

NameDescription

Reproducibility Project: Psychology

A collaborative effort published in 2015 that attempted to replicate a sample of psychological studies and stimulated discussion about research reproducibility.

Registered Reports

A publication format in which research questions and methods can receive peer review before the results are known.

Open Science Framework

Provides infrastructure for sharing research materials, data, analysis plans and other research outputs.

Limitations

  • Exact replication may be difficult when procedures are incompletely documented.

  • Different populations and contexts may produce different outcomes.

  • A failed replication does not automatically prove the original finding was false.

  • Successful replication does not guarantee that a theory is universally correct.

Deep Understanding

What

Replication and reproducibility examine whether scientific results are stable and whether analytical findings can be independently checked.

Why

Individual studies may produce findings that are affected by chance, sampling variation, measurement problems or undisclosed analytical decisions.

How

Researchers share methods, data and code where ethically and legally appropriate, repeat procedures and compare the resulting findings with previous reports.

When

These practices are especially important for influential findings, disputed results, high-impact interventions and research used to guide policy or practice.

Example

A study reports that a particular learning technique improves memory. Independent researchers repeat the investigation with different participants to evaluate whether the finding is robust.

Comparison

Replication

Collects new evidence to examine an earlier finding.

Reproducibility

Re-executes an existing analysis using the original research materials.

Generalization

Examines whether findings apply to different populations or contexts.

Mathematical Coverage

Relevance

Replication research uses statistical methods to compare effect estimates, quantify uncertainty and evaluate consistency across studies.

Methods

NameDescription

Effect-size comparison

Compares the magnitude and direction of effects across original and replication studies.

Confidence intervals

Help evaluate the uncertainty of estimates in each study.

Meta-analysis

Combines effect estimates from multiple studies when the studies are sufficiently comparable.

Heterogeneity analysis

Examines how much study results differ beyond what would be expected from sampling error alone under the model.

Worked Example

Problem

An original study estimates a standardized effect size of d = 0.40, while a replication estimates d = 0.35.

Interpretation

The estimates are similar in direction and magnitude, but their uncertainty and study quality must be examined before concluding that the finding has replicated.

Important Note

A difference between effect estimates does not automatically indicate failure. Sampling error, population differences and methodological variation must be considered.

Caution

Comparing p-values alone is not a sufficient method for evaluating replication. Effect sizes, confidence intervals, design and measurement quality are also important.

Practical Learning

Applications

  • Evaluating the strength of psychological findings before using them in practice.

  • Checking whether a published analysis can be reproduced.

  • Improving transparency in student research projects.

  • Building cumulative evidence through collaborative studies.

  • Assessing the robustness of educational and behavioural interventions.

Case Study

Title

Replicating a memory experiment

Scenario

A published study reports improved memory performance after a particular study technique.

Procedure

An independent research team reviews the original method, defines the target effect and conducts a new investigation with a suitable sample.

Interpretation

Similar results provide additional support, while different results may reveal uncertainty, contextual differences or methodological problems requiring further investigation.

Exercise

Imagine that an important psychological finding cannot be reproduced. List at least four possible explanations other than deliberate research misconduct.

Critical Thinking

Misconceptions

ClaimCorrection

A failed replication proves that the original researchers were dishonest.

Differences can result from chance, measurement, sampling, context or methodological variation.

A successful replication proves that a finding is universally true.

Replication strengthens confidence but does not establish universal applicability.

Reproducibility and replication mean exactly the same thing.

They are related but commonly refer to different procedures.

Debates

  • How closely should a replication reproduce the original study?

  • Should conceptual replications receive the same priority as direct replications?

  • How can researchers balance transparency with participant privacy?

Limitations

  • Research materials may be unavailable.

  • Some studies cannot be repeated ethically or practically.

  • Contextual differences may affect findings.

  • Publication incentives may discourage replication studies.

Critical Questions

  • What should count as a successful replication?

  • Why is a replication's effect size important?

  • How can open research practices improve psychological science?

Knowledge Assessment

Mcq

QuestionOptionsAnswerExplanation

What is the main purpose of replication?

  • To guarantee that a theory is permanently true

  • To examine whether previous findings can be observed again

  • To eliminate the need for independent research

  • To avoid collecting new data

To examine whether previous findings can be observed again

Replication provides new evidence about the stability of previous findings.

Which activity most directly represents computational reproducibility?

  • Interviewing a new group of participants

  • Repeating the same statistical analysis using the original data and code

  • Writing a different theoretical explanation

  • Changing the research question

Repeating the same statistical analysis using the original data and code

Computational reproducibility checks whether the reported analysis can be regenerated.

Analytical Questions

  • Differentiate direct replication, conceptual replication and computational reproducibility.

  • Explain why replication is important for psychological knowledge.

  • Discuss possible reasons why a study may fail to replicate.

Application Exercises

  • Prepare a replication plan for a simple memory experiment.

  • Identify which research materials should be shared to support computational reproducibility.

Answer Guidance

Answers should distinguish repeating an analysis from conducting a new study and should avoid interpreting replication failure as automatic proof of error or misconduct.

Further Learning

Prerequisites

  • Empirical Evidence and Psychological Knowledge

  • Reliability and Validity of Psychological Knowledge

  • Hypotheses and Scientific Predictions

Related Concepts

  • Research Transparency

  • Preregistration

  • Meta-analysis

  • Scientific Reasoning and Critical Thinking

Next Learning Path

Study open science, research reporting, statistical power, meta-analysis and research integrity.

Sources and verification

Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.

  • Source verification pending; check an authoritative textbook, university, standards body, or peer-reviewed source.

Suggest a correction or source · More in Psychology