Causality, Validity, and Reliability

Causality is a relationship between 2 events in which 1 event causes the other. Simply because relationships are observed between 2 variables (i.e., associations or correlations) does not imply that one variable actually caused the outcome. Demonstrating causality between an exposure and an outcome is the main objective of most published medical research. To ensure causality exists and is not an artifact of a flawed study design or other factors, various criteria must be met while showing the reproducibility (reliability), internal congruence (internal validity), and generalizability (external validity) of the study.

Last update:

Table of Contents

Share this concept:

Share on facebook
Share on twitter
Share on linkedin
Share on reddit
Share on email
Share on whatsapp

Causality

Definition

Causality is the relationship between cause and effect.

Principles

  • The principle of causality is that all events have a cause. 
  • Indicates a logical relationship between 2 events (a cause and an effect) and an order between them (the cause precedes the effect)
  • In medicine, establishing causality:
    • Helps identify the cause of a disease
    • Enables the best possible management of the patient
    • Allows researchers to develop the best diagnostic tests

Causality versus correlation

“Correlation is not causation.”

  • Causality means that 1 event was caused by another vent
  • Correlation (or association) means that 2 things are connected, but it does not imply causality.

Example:

Below is a graph depicting the relationship between drowning deaths and eating ice cream. As ice cream consumption goes up, so do drowning deaths. However, this study is only showing a correlation rather than causation. Eating ice cream does not cause drowning deaths. Rather, on hot days, people are more likely to eat ice cream, and they are more likely to go to the beach and drown. Thus, temperature is a confounding factor, leading to an observed relationship when in reality there is no causality.

Example graph showing a correlation between events (rather than causation).

Example graph showing a correlation between events (rather than causation)

Image by Lecturio. License: CC BY-NC-SA 4.0

Bradford Hill criteria

Background:

  • Also known as Hill’s criteria for causation
  • A group of 9 principles useful in establishing epidemiologic evidence of a causal relationship between a presumed cause and an observed effect
  • If a majority of the principles are satisfied, the relationship between variables is likely causal.
  • Widely used in public health research

The 9 principles:

  1. Strength: What is the effect size or how big is the relationship? How large is the relative risk or odds ratio?
  2. Consistency: Do I see the relationship frequently? Is it reproducible?
  3. Specificity: Does this exposure exclusively cause this outcome?
  4. Temporality: Does the exposure come before the outcome? 
  5. Biological gradient/dose-response relationship: 
  • Does more of the exposure cause more of the outcome?
  • Does removal of the exposure decrease risk of the outcome?
  1. Plausibility: Does the relationship make sense biologically?
  2. Coherence: Does the observed relationship fit into the general knowledge of science and medicine?
  3. Experiment: Can a randomized controlled experiment be conducted (in humans or animals)?
  4. Analogy: 
  • Can we devise an analogous relationship between the outcome and another exposure?
  • Have I explored other theoretical possibilities for the observed relationship?

Example: Applying the Bradford Hill criteria to establish causality

As seen in the table below, a majority of the principles are satisfied, so you can be reasonably sure that smoking actually causes lung cancer.

Table: Applying the Bradford Hill criteria: Does smoking cause lung cancer?
Principle Principle satisfied Explanation
Strength Yes There is a strong relative risk (RR) association between smoking and lung cancer.
Consistency Yes This ↑↑ RR has been reproduced across many cohort studies
Specificity No Smoking can lead to many different outcomes, and other exposures can also lead to lung cancer.
Temporality Yes Smoking precedes the development of lung cancer in the vast majority of cases.
Biological gradient Yes The more you smoke, the higher your RR of lung cancer.
Plausibility Yes In the lab, it has been shown that lung tissue exposed to the carcinogens found in cigarette smoke shows an increase in genetic mutations.
Coherence Yes Certain chemicals within cigarette smoke are carcinogens and thus ↑ risk for lung cancer: this idea fits in with our larger understanding of medicine and science
Experiment Yes We have exposed laboratory animals to smoke, and they have developed cancer.
Analogy Not really Other options have been explored, and there may be other potential possibilities.

Causal Relationships

Definition

A causal relation between 2 events exists if the occurrence of the 1st event causes the 2nd event. 

Principles

  • The 1st event is then referred to as a cause and the 2nd event the effect. 
  • A correlation between 2 events does not imply causation
  • However, if there is a causal relationship between 2 events, they will be correlated.
  • A causal pathway (the pathway from the cause to the effect) can be:
    • Direct: The factor causes the disease without intermediary steps.
    • Indirect: The factor causes the disease but only through 1 or more intermediary steps.

Types of causal relationships

There are 4 types of causal relationships or factors based on whether or not the exposure was necessary to develop the outcome, and whether exposure is sufficient on its own to cause the outcome. These 4 types are:

  1. Necessary and sufficient
  2. Necessary but not sufficient
  3. Sufficient but not necessary
  4. Neither necessary nor sufficient

Example #1: Necessary and sufficient

  • Necessary: It is impossible to have the outcome without the exposure.
  • Sufficient: It is all that is needed to produce the outcome.
  • Example: Infection with coronavirus is both necessary and sufficient to cause the disease SARS. 
A diagram of a causal factor that is necessary and sufficient

A diagram of a causal factor that is necessary and sufficient

Image by Lecturio. License: CC BY-NC-SA 4.0

Example #2: Necessary and not sufficient

  • Necessary: The exposure is required to develop the outcome.
  • Not sufficient:
    • The exposure needs to be aided by some other factor in order to produce the outcome.
    • Individual factors cannot produce the outcome by themselves. 
  • Example: A disease is caused by a gene that becomes activated by a particular environmental trigger. Both the gene and environmental trigger are necessary for the disease, but neither is sufficient alone to cause the disease.
A diagram of a causal factor that is necessary but not sufficient

A diagram of a causal factor that is necessary but not sufficient

Image by Lecturio. License: CC BY-NC-SA 4.0

Example #3: Sufficient and not necessary

  • Sufficient: The exposure alone can produce the outcome.
  • Not necessary: The exposure is not the only one that can produce the outcome.
  • Example: Both radiation poisoning alone and benzene poisoning alone are sufficient to cause leukemia. Therefore, it is not necessary to have radiation exposure in order to develop leukemia (because you could have been exposed to benzene instead). Thus, radiation poisoning and benzene poisoning are both sufficient but not necessary for the development of leukemia.
A diagram of causal factors that are sufficient but not necessary

A diagram of causal factors that are sufficient but not necessary

Image by Lecturio. License: CC BY-NC-SA 4.0

Example #4: Neither necessary nor sufficient

  • Not necessary: Several factors or exposures have complex interactions that produce the outcome.
  • Not sufficient: The individual exposures are not enough to produce the outcome alone.
  • Example: prostate cancer. There are multiple risk factors that individually are neither necessary nor sufficient to cause prostate cancer alone. There are multiple combinations of exposures possible, making them all neither necessary nor sufficient. 
  • This is arguably the most common type of causal relationship encountered in clinical practice.
A diagram of causal factors that are neither necessary nor sufficient

A diagram of causal factors that are neither necessary nor sufficient

Image by Lecturio. License: CC BY-NC-SA 4.0

Reliability and Validity

Reliability refers to the reproducibility of a test or research finding: Is the test or finding repeatable?

  • Reliability = reproducibility/consistency/precision
  • ↑ Reliability leads to ↓ random error rates
  • Reliability improves with ↓ standard deviation and ↑ power

Validity refers to how accurate a test or research finding is: Are the results representative of the real world?

  • Validity = accuracy
    • Results reflect reality and can be believed.
    • Validity depends on the elimination of biases.
    • Sensitivity and specificity are measures of validity.
  • Internal validity: 
    • The causal relationships are meaningful within the context of the study. 
    • Requirements for internal validity:
      • Temporality
      • Strength
      • Plausibility
  • External validity or generalizability: Results can be applied to other patients or settings.

Note: An invalid study can still be reliable, but an unreliable study cannot be valid. In other words, a relationship that does not represent the real world (invalid) can be seen multiple times in a study (reliable), but a study that cannot be reproduced (unreliable) cannot represent the real world (validity).

Reliability and validity

Reliability and validity

Image by Lecturio. License: CC BY-NC-SA 4.0

Reliability measurements

  • Reliability is measured quantitatively with a coefficient, typically written as r.
  • R is valued between 0 and 1:
    • r = 1: perfectly reliable test
    • r = 0: complete absence of reliability
  • r = ↑ reliability = ↓ errors
  • Investigators typically want r at least 0.9, which means 90% of the data are accurate while 10% is caused by errors.

Threats to reliability and validity

Threats to reliability:

  • Poor sampling strategies
    • Example: You want to measure the average age in a community. You might go to a retirement center and find an average age of 72 or go to a high school and find an average age of 16. These samples are not representative of the population you are trying to study, so your data are not reliable. 
  • Instability in the thing being measured
    • Example: You want to measure blood pressure, but blood pressure changes throughout the day, based on factors such as activity level and how horizontal you are (lying down versus standing). If these confounding variables are not accounted for, your data will not be reliable.
  • Divergences between observers, especially in cases where data collection requires a qualitative assessment by the observer
    • Example: If you’re asking observers to assess mood, different observers may have different opinions about how to score different moods. This would decrease reliability.

Threats to internal validity:

  • Confounding factors: a variable that creates an artificial relationship or that masks a real relationship between study variables (see the example of ice cream and drowning deaths above)
  • Selection bias: the error introduced when the study population does not represent the target population due to some selection preference (see Types of Biases for details)

Threats to external validity:

  • Too many exclusion criteria (overly specific study characteristics that do not represent other populations)
  • Hawthorne effect (observer effect): People in studies change their behavior because they are being watched.
  • Rosenthal effect: The investigator’s expectations about the outcome of a given study affect the actual study outcome.

References

  1. Celentano, D, Szklo, M. (2019). From association to causation: Deriving inferences from epidemiologic studies.
  2. Everitt, BS, Skrondal, A. (2010). The Cambridge Dictionary of Statistics, Cambridge University Press.
  3. Redmond, CK, Colton, T. (2001) Biostatistics in Clinical Trial, 2001: p. 522.
  4. Greenberg, RS. (2015). Medical epidemiology. In Population Health and Effective Health Care (5th ed.).
  5. Kanchanaraksa S. (2008). Evaluation of diagnostic and screening Tests: Validity and reliability. In The Johns Hopkins University Bloomberg School of Public Health.
  6. Höfler, M. (2005). The Bradford Hill considerations on causality: A counterfactual perspective? Emerging Themes in Epidemiology, 2 (1): 11.

Study on the Go

Lecturio Medical complements your studies with evidence-based learning strategies, video lectures, quiz questions, and more – all combined in one easy-to-use resource.

Learn even more with Lecturio:

Complement your med school studies with Lecturio’s all-in-one study companion, delivered with evidence-based learning strategies.

🍪 Lecturio is using cookies to improve your user experience. By continuing use of our service you agree upon our Data Privacy Statement.

Details