Causality, Validity, and Reliability

Causality

Definition

Causality is the relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship between cause and effect.

Principles

• The principle of causality is that all events have a cause. 
• Indicates a logical relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship between 2 events (a cause and an effect) and an order between them (the cause precedes the effect)
• In medicine, establishing causality:
  • Helps identify the cause of a disease
  • Enables the best possible management of the patient
  • Allows researchers to develop the best diagnostic tests Diagnostic tests Diagnostic tests are important aspects in making a diagnosis. Some of the most important epidemiological values of diagnostic tests include sensitivity and specificity, false positives and false negatives, positive and negative predictive values, likelihood ratios, and pre-test and post-test probabilities. Epidemiological Values of Diagnostic Tests

Causality versus correlation

“Correlation is not causation.”

• Causality means that 1 event was caused by another vent
• Correlation (or association) means that 2 things are connected, but it does not imply causality.

Example:

Below is a graph depicting the relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship between drowning Drowning Drowning occurs due to respiratory impairment from submersion or immersion in a liquid medium. Aspiration of water leads to hypoxemia, which affects all organ systems, resulting in respiratory insufficiency and acute respiratory distress syndrome (ARDS), cardiac arrhythmias, and neuronal damage. Drowning deaths and eating ice cream. As ice cream consumption goes up, so do drowning Drowning Drowning occurs due to respiratory impairment from submersion or immersion in a liquid medium. Aspiration of water leads to hypoxemia, which affects all organ systems, resulting in respiratory insufficiency and acute respiratory distress syndrome (ARDS), cardiac arrhythmias, and neuronal damage. Drowning deaths. However, this study is only showing a correlation rather than causation. Eating ice cream does not cause drowning Drowning Drowning occurs due to respiratory impairment from submersion or immersion in a liquid medium. Aspiration of water leads to hypoxemia, which affects all organ systems, resulting in respiratory insufficiency and acute respiratory distress syndrome (ARDS), cardiac arrhythmias, and neuronal damage. Drowning deaths. Rather, on hot days, people are more likely to eat ice cream, and they are more likely to go to the beach and drown. Thus, temperature is a confounding factor, leading to an observed relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship when in reality there is no causality.

Bradford Hill criteria

Background:

• Also known as Hill’s criteria for causation
• A group of 9 principles useful in establishing epidemiologic evidence of a causal relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship between a presumed cause and an observed effect
• If a majority of the principles are satisfied, the relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship between variables is likely causal.
• Widely used in public health research Research Critical and exhaustive investigation or experimentation, having for its aim the discovery of new facts and their correct interpretation, the revision of accepted conclusions, theories, or laws in the light of newly discovered facts, or the practical application of such new or revised conclusions, theories, or laws. Conflict of Interest

The 9 principles:

1. Strength: What is the effect size Effect size Effect size is the standardized mean difference between 2 groups, which is exactly equivalent to the “Z-score” of a standard normal distribution. Statistical Power or how big is the relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship? How large is the relative risk Relative risk Relative risk (RR) is the risk of a disease or condition occurring in a group or population with a particular exposure relative to a control (unexposed) group. Measures of Risk or odds ratio Odds ratio The ratio of two odds. The exposure-odds ratio for case control data is the ratio of the odds in favor of exposure among cases to the odds in favor of exposure among noncases. The disease-odds ratio for a cohort or cross section is the ratio of the odds in favor of disease among the exposed to the odds in favor of disease among the unexposed. The prevalence-odds ratio refers to an odds ratio derived cross-sectionally from studies of prevalent cases. Measures of Risk?
2. Consistency Consistency Dermatologic Examination: Do I see the relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship frequently? Is it reproducible?
3. Specificity: Does this exposure exclusively cause this outcome?
4. Temporality: Does the exposure come before the outcome? 
5. Biological gradient/dose-response relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship: 

• Does more of the exposure cause more of the outcome?
• Does removal of the exposure decrease risk of the outcome?

6. Plausibility: Does the relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship make sense biologically?
7. Coherence Coherence A view of the world and the individual’s environment as comprehensible, manageable, and meaningful, claiming that the way people view their life has a positive influence on their health. Neurological Examination: Does the observed relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship fit into the general knowledge of science and medicine?
8. Experiment: Can a randomized controlled experiment be conducted (in humans or animals Animals Unicellular or multicellular, heterotrophic organisms, that have sensation and the power of voluntary movement. Under the older five kingdom paradigm, animalia was one of the kingdoms. Under the modern three domain model, animalia represents one of the many groups in the domain eukaryota. Cell Types: Eukaryotic versus Prokaryotic)?
9. Analogy: 

• Can we devise an analogous relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship between the outcome and another exposure?
• Have I explored other theoretical possibilities for the observed relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship?

Example: Applying the Bradford Hill criteria to establish causality

As seen in the table below, a majority of the principles are satisfied, so you can be reasonably sure that smoking Smoking Willful or deliberate act of inhaling and exhaling smoke from burning substances or agents held by hand. Interstitial Lung Diseases actually causes lung cancer Lung cancer Lung cancer is the malignant transformation of lung tissue and the leading cause of cancer-related deaths. The majority of cases are associated with long-term smoking. The disease is generally classified histologically as either small cell lung cancer or non-small cell lung cancer. Symptoms include cough, dyspnea, weight loss, and chest discomfort. Lung Cancer.

Causal Relationships

Definition

A causal relation between 2 events exists if the occurrence of the 1st event causes the 2nd event. 

Principles

• The 1st event is then referred to as a cause and the 2nd event the effect. 
• A correlation between 2 events does not imply causation. 
• However, if there is a causal relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship between 2 events, they will be correlated.
• A causal pathway (the pathway from the cause to the effect) can be:
  • Direct: The factor causes the disease without intermediary steps.
  • Indirect: The factor causes the disease but only through 1 or more intermediary steps.

Types of causal relationships

There are 4 types of causal relationships or factors based on whether or not the exposure was necessary to develop the outcome, and whether exposure is sufficient on its own to cause the outcome. These 4 types are:

1. Necessary and sufficient
2. Necessary but not sufficient
3. Sufficient but not necessary
4. Neither necessary nor sufficient

Example #1: Necessary and sufficient

• Necessary: It is impossible to have the outcome without the exposure.
• Sufficient: It is all that is needed to produce the outcome.
• Example: Infection with coronavirus Coronavirus Coronaviruses are a group of related viruses that contain positive-sense, single-stranded RNA. Coronavirus derives its name from “κορώνη korṓnē” in Greek, which translates as “crown,” after the small club-shaped proteins visible as a ring around the viral envelope in electron micrographs. Coronavirus is both necessary and sufficient to cause the disease SARS. 

Example #2: Necessary and not sufficient

• Necessary: The exposure is required to develop the outcome.
• Not sufficient:
  • The exposure needs to be aided by some other factor in order to produce the outcome.
  • Individual factors cannot produce the outcome by themselves. 
• Example: A disease is caused by a gene Gene A category of nucleic acid sequences that function as units of heredity and which code for the basic instructions for the development, reproduction, and maintenance of organisms. Basic Terms of Genetics that becomes activated by a particular environmental trigger Trigger The type of signal that initiates the inspiratory phase by the ventilator Invasive Mechanical Ventilation. Both the gene Gene A category of nucleic acid sequences that function as units of heredity and which code for the basic instructions for the development, reproduction, and maintenance of organisms. Basic Terms of Genetics and environmental trigger Trigger The type of signal that initiates the inspiratory phase by the ventilator Invasive Mechanical Ventilation are necessary for the disease, but neither is sufficient alone to cause the disease.

Example #3: Sufficient and not necessary

• Sufficient: The exposure alone can produce the outcome.
• Not necessary: The exposure is not the only one that can produce the outcome.
• Example: Both radiation Radiation Emission or propagation of acoustic waves (sound), electromagnetic energy waves (such as light; radio waves; gamma rays; or x-rays), or a stream of subatomic particles (such as electrons; neutrons; protons; or alpha particles). Osteosarcoma poisoning alone and benzene poisoning alone are sufficient to cause leukemia. Therefore, it is not necessary to have radiation Radiation Emission or propagation of acoustic waves (sound), electromagnetic energy waves (such as light; radio waves; gamma rays; or x-rays), or a stream of subatomic particles (such as electrons; neutrons; protons; or alpha particles). Osteosarcoma exposure in order to develop leukemia (because you could have been exposed to benzene instead). Thus, radiation Radiation Emission or propagation of acoustic waves (sound), electromagnetic energy waves (such as light; radio waves; gamma rays; or x-rays), or a stream of subatomic particles (such as electrons; neutrons; protons; or alpha particles). Osteosarcoma poisoning and benzene poisoning are both sufficient but not necessary for the development of leukemia.

Example #4: Neither necessary nor sufficient

• Not necessary: Several factors or exposures have complex interactions that produce the outcome.
• Not sufficient: The individual exposures are not enough to produce the outcome alone.
• Example: prostate Prostate The prostate is a gland in the male reproductive system. The gland surrounds the bladder neck and a portion of the urethra. The prostate is an exocrine gland that produces a weakly acidic secretion, which accounts for roughly 20% of the seminal fluid. cancer. There are multiple risk factors that individually are neither necessary nor sufficient to cause prostate Prostate The prostate is a gland in the male reproductive system. The gland surrounds the bladder neck and a portion of the urethra. The prostate is an exocrine gland that produces a weakly acidic secretion, which accounts for roughly 20% of the seminal fluid. cancer alone. There are multiple combinations of exposures possible, making them all neither necessary nor sufficient. 
• This is arguably the most common type of causal relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship encountered in clinical practice.

Reliability and Validity

Reliability refers to the reproducibility of a test or research Research Critical and exhaustive investigation or experimentation, having for its aim the discovery of new facts and their correct interpretation, the revision of accepted conclusions, theories, or laws in the light of newly discovered facts, or the practical application of such new or revised conclusions, theories, or laws. Conflict of Interest finding: Is the test or finding repeatable?

• Reliability = reproducibility/ consistency Consistency Dermatologic Examination/precision
• ↑ Reliability leads to ↓ random error Error Refers to any act of commission (doing something wrong) or omission (failing to do something right) that exposes patients to potentially hazardous situations. Disclosure of Information rates
• Reliability improves with ↓ standard deviation Standard deviation The standard deviation (SD) is a measure of how far each observed value is from the mean in a data set. Measures of Central Tendency and Dispersion and ↑ power

Validity refers to how accurate a test or research Research Critical and exhaustive investigation or experimentation, having for its aim the discovery of new facts and their correct interpretation, the revision of accepted conclusions, theories, or laws in the light of newly discovered facts, or the practical application of such new or revised conclusions, theories, or laws. Conflict of Interest finding is: Are the results representative of the real world?

• Validity = accuracy
  • Results reflect reality and can be believed.
  • Validity depends on the elimination Elimination The initial damage and destruction of tumor cells by innate and adaptive immunity. Completion of the phase means no cancer growth. Cancer Immunotherapy of biases.
  • Sensitivity and specificity Sensitivity and Specificity Binary classification measures to assess test results. Sensitivity or recall rate is the proportion of true positives. Specificity is the probability of correctly determining the absence of a condition. Epidemiological Values of Diagnostic Tests are measures of validity.
• Internal validity: 
  • The causal relationships are meaningful within the context of the study. 
  • Requirements for internal validity:
    • Temporality
    • Strength
    • Plausibility
• External validity or generalizability: Results can be applied to other patients Patients Individuals participating in the health care system for the purpose of receiving therapeutic, diagnostic, or preventive procedures. Clinician–Patient Relationship or settings.

Note: An invalid study can still be reliable, but an unreliable study cannot be valid. In other words, a relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship that does not represent the real world (invalid) can be seen multiple times in a study (reliable), but a study that cannot be reproduced (unreliable) cannot represent the real world (validity).

Reliability measurements

• Reliability is measured quantitatively with a coefficient, typically written as r.
• R is valued between 0 and 1:
  • r = 1: perfectly reliable test
  • r = 0: complete absence of reliability
• ↑ r = ↑ reliability = ↓ errors
• Investigators typically want r at least 0.9, which means 90% of the data are accurate while 10% is caused by errors.

Threats to reliability and validity

Threats to reliability:

• Poor sampling strategies
  • Example: You want to measure the average age in a community. You might go to a retirement center and find an average age of 72 or go to a high school and find an average age of 16. These samples are not representative of the population you are trying to study, so your data are not reliable. 
• Instability in the thing being measured
  • Example: You want to measure blood pressure, but blood pressure changes throughout the day, based on factors such as activity level and how horizontal you are (lying down versus standing). If these confounding variables Confounding variables A confound is an additional variable other than the independent variable that has an effect on the dependent variable, causing an erroneous relationship to be inferred between them. Types of Biases are not accounted for, your data will not be reliable.
• Divergences between observers, especially in cases where data collection requires a qualitative assessment by the observer
  • Example: If you’re asking observers to assess mood, different observers may have different opinions about how to score different moods. This would decrease reliability.

Threats to internal validity:

• Confounding factors: a variable Variable Variables represent information about something that can change. The design of the measurement scales, or of the methods for obtaining information, will determine the data gathered and the characteristics of that data. As a result, a variable can be qualitative or quantitative, and may be further classified into subgroups. Types of Variables that creates an artificial relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship or that masks a real relationship Relationship A connection, association, or involvement between 2 or more parties. Clinician–Patient Relationship between study variables (see the example of ice cream and drowning Drowning Drowning occurs due to respiratory impairment from submersion or immersion in a liquid medium. Aspiration of water leads to hypoxemia, which affects all organ systems, resulting in respiratory insufficiency and acute respiratory distress syndrome (ARDS), cardiac arrhythmias, and neuronal damage. Drowning deaths above)
• Selection Selection Lymphocyte activation by a specific antigen thus triggering clonal expansion of lymphocytes already capable of mounting an immune response to the antigen. B cells: Types and Functions bias Bias Epidemiological studies are designed to evaluate a hypothesized relationship between an exposure and an outcome; however, the existence and/or magnitude of these relationships may be erroneously affected by the design and execution of the study itself or by conscious or unconscious errors perpetrated by the investigators or the subjects. These systematic errors are called biases. Types of Biases: the error Error Refers to any act of commission (doing something wrong) or omission (failing to do something right) that exposes patients to potentially hazardous situations. Disclosure of Information introduced when the study population does not represent the target population due to some selection Selection Lymphocyte activation by a specific antigen thus triggering clonal expansion of lymphocytes already capable of mounting an immune response to the antigen. B cells: Types and Functions preference (see Types of Biases for details)

Threats to external validity:

• Too many exclusion criteria (overly specific study characteristics that do not represent other populations)
• Hawthorne effect Hawthorne effect Refers to the tendency of subjects in a study to behave or act differently (i.e., work harder) when they know they are being watched. Types of Biases (observer effect): People in studies change their behavior because they are being watched.
• Rosenthal effect Rosenthal effect Refers to the tendency of subjects or investigators to behave differently based on other’s expectations. Types of Biases: The investigator’s expectations about the outcome of a given study affect Affect The feeling-tone accompaniment of an idea or mental representation. It is the most direct psychic derivative of instinct and the psychic representative of the various bodily changes by means of which instincts manifest themselves. Psychiatric Assessment the actual study outcome.