✦ScienexySCIENCE KNOWLEDGE LIBRARY
learning-resource · SCI-PSY-002-135

Research Bias and Sources of Error

Research bias and error are influences that can distort psychological measurements, participant selection, data collection, analysis or interpretation. Understanding these problems helps researchers design stronger studies, evaluate evidence critically and avoid conclusions that exceed the quality of the data.

Explanation

A fuller explanation is being developed. Verify this concept with cited sources before relying on it.

Example or application

A worked example has not yet been added.

Conceptual Coverage

Bias

A systematic influence that shifts research procedures or results in a particular direction.

Error

A difference between an observed measurement or estimate and the value it is intended to represent.

Major Distinctions

Systematic Error

A consistent distortion caused by measurement, selection, procedure or analysis.

Random Error

Unpredictable variation that may increase uncertainty and reduce precision.

Bias

Often refers to systematic distortion, although the exact use depends on the research context.

Major Sources

Sampling Bias

The selection process produces a sample that systematically differs from the intended population.

Selection Bias

Entry into a study or analysis differs systematically across relevant participant characteristics.

Measurement Bias

An instrument or procedure systematically measures a construct inaccurately.

Observer Bias

Researcher expectations influence observation or recording.

Interviewer Bias

An interviewer's behaviour or question delivery influences responses.

Response Bias

Participants answer in ways that do not accurately represent their experiences or behaviour.

Social Desirability Bias

Participants report answers they believe are socially acceptable.

Recall Bias

Participants remember past events inaccurately or differently across groups.

Publication Bias

Studies with particular results may be more likely to be published or made visible.

Confirmation Bias

Researchers or readers favour evidence consistent with existing beliefs.

Data Related Problems

  • Missing data.

  • Data-entry errors.

  • Outliers.

  • Inconsistent coding.

  • Inappropriate statistical assumptions.

  • Selective outcome reporting.

Scientific Foundation

Methodological Importance

Bias evaluation is central to research design, measurement theory, evidence synthesis and scientific interpretation.

Historical Development

The development of experimental controls, randomisation, blinding, standardised measurement and statistical quality assessment reflects continuing efforts to reduce research bias.

Scientific Principles

  • Plan methods before examining results when appropriate.

  • Use standardised and validated measures.

  • Apply suitable randomisation and blinding where feasible.

  • Document missing data and exclusions.

  • Report all relevant outcomes and analytical decisions.

  • Consider alternative explanations and contradictory evidence.

Open Science

Preregistration, transparent reporting, accessible methods and independent replication can reduce opportunities for selective analysis and improve scrutiny, although none eliminates every source of bias.

Deep Understanding

Bias Versus Random Error

Random error generally increases uncertainty and reduces precision, while systematic bias can shift results consistently away from the value being estimated. Increasing sample size can reduce some random sampling error but does not automatically remove systematic bias.

Selection Mechanism

Bias can arise before data collection when recruitment excludes relevant participants or when participation depends on factors related to the research question.

Measurement Mechanism

A measure may systematically overestimate or underestimate a construct because of poor calibration, leading wording, inconsistent administration or cultural mismatch.

Analysis Mechanism

Selective exclusion, inappropriate statistical methods, multiple unreported analyses and selective outcome reporting can distort the apparent strength of evidence.

Researcher Influence

Expectations can influence research questions, measurement choices, data interpretation and reporting. Transparent procedures and independent scrutiny help limit these influences.

Combined Bias

Different sources of bias can interact. For example, selective recruitment and inaccurate self-report may jointly distort an estimate.

Mathematical Coverage

Measurement Error

Model

Observed score = True score + Measurement error

Explanation

In classical test theory, an observed score is conceptualised as a true-score component plus error under the model's assumptions.

Absolute Error

Absolute error = |Observed value − Reference value|, when a suitable reference value exists.

Relative Error

Relative error = Absolute error / |Reference value|, when the reference value is non-zero and the comparison is meaningful.

Worked Example

Reference Value

50

Observed Value

54

Absolute Error

|54 − 50| = 4.

Relative Error

4 / 50 = 0.08 = 8%.

Interpretation

The measurement differs from the reference value by 4 units, or 8% of the reference value.

Missing Data

The percentage missing can be calculated as (Missing observations / Expected observations) × 100, but the impact depends on the mechanism generating missingness.

Statistical Caution

A numerical adjustment cannot reliably remove bias when the source and structure of the bias are not understood.

Practical Learning

Example Study

Question

What proportion of students experience frequent study interruptions?

Possible Biases

  • A convenience sample recruited from one classroom may not represent all students.

  • Participants may underreport interruptions because they feel embarrassed.

  • A vague question may be interpreted differently by participants.

  • Nonrespondents may have different study habits from respondents.

Improvements

  • Recruit from an appropriately defined population.

  • Use neutral and clearly worded questions.

  • Pilot-test the questionnaire.

  • Report response rates and recruitment procedures.

  • Evaluate the potential impact of missing information.

Applications

  • Evaluating psychological surveys.

  • Assessing clinical research evidence.

  • Reviewing educational intervention studies.

  • Interpreting observational research.

  • Conducting systematic reviews and meta-analyses.

Exercise

Identify three possible sources of bias in a study that recruits only volunteers from an online student discussion group.

Critical Thinking

Misconceptions

  • Bias is not always intentional misconduct.

  • A large sample cannot automatically correct systematic selection bias.

  • Randomisation reduces certain biases but does not remove all problems.

  • Blinding is useful only where it is feasible and relevant to the study.

  • A statistically significant result can still be biased.

Limitations

  • Some bias cannot be measured directly.

  • Researchers may not know all relevant confounders.

  • Methods that reduce one bias may introduce other practical limitations.

  • Retrospective bias assessment may be incomplete.

  • Publication and reporting practices can distort the available evidence base.

Evaluation Questions

  • How were participants selected?

  • Were measurements standardised and appropriate?

  • Could expectations have influenced observations or responses?

  • Were exclusions and missing data reported?

  • Were all relevant results and limitations disclosed?

Knowledge Assessment

Mcqs

QuestionOptionsAnswerExplanation

Which type of error produces a consistent directional distortion?

  • Systematic error

  • Pure random variation

  • Sampling fluctuation only

  • Unavoidable numerical rounding

Systematic error

Systematic error consistently shifts measurements or estimates away from the intended value.

What is social desirability bias?

  • Reporting answers perceived as socially acceptable

  • Randomly selecting participants

  • Using multiple measurement instruments

  • Repeating an experiment independently

Reporting answers perceived as socially acceptable

Participants may alter their responses to appear more acceptable to others.

Analytical Questions

  • How does systematic error differ from random error?

  • Why does increasing sample size not necessarily eliminate bias?

  • How can preregistration improve research transparency?

Application Exercise

Task

A survey about study habits is distributed only to students who attend an optional study-skills workshop.

Possible Problems

  • The sample may overrepresent students already interested in improving their study habits.

  • The results may not represent students who did not attend.

  • Participation may be related to the outcome being investigated.

Improvement

Recruit from a broader and clearly defined student population using a suitable sampling approach.

Further Learning

Prerequisites

  • Sampling and population.

  • Measurement and operational definitions.

  • Research design.

Related Topics

  • Validity and Reliability

  • Research Ethics

  • Statistical Inference

  • Experimental Controls

  • Open Science

  • Systematic Reviews

Advanced Directions

  • Risk-of-bias assessment.

  • Missing-data mechanisms.

  • Sensitivity analysis.

  • Publication bias and funnel plots.

  • Confounding and causal diagrams.

Learning Path

Learn the distinction between systematic and random error, study major sources of bias, practise identifying them in research examples, and then progress to bias assessment and statistical sensitivity methods.

Sources and verification

Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.

  • Source verification pending; check an authoritative textbook, university, standards body, or peer-reviewed source.

Suggest a correction or source · More in Foundations of Psychology