Characteristics of Experimental Research and Measurement Fundamentals

Workshop and Software Resources

• Workshops begin following this session.

• The software required for these workshops is jamovi, which is available for download at https://www.jamovi.org/download.html.

• jamovi is freely available under the AGPL3 license, meaning it is free to use for any purpose, including institutional use.

• Available versions of jamovi for different operating systems:

  • Windows: Version $2.0.0$ (Current, available as .exe or .zip), Version $1.6.23$ (Solid, available as .exe or .zip), and Version $1.2.27$ (Legacy, available as .exe or .zip).
  • macOS: Version $2.0.0$ (Current), Version $1.6.23$ (Solid, compatible with macOS $10.13+$), Version $1.2.27$ (Legacy, compatible with macOS $10.11+$), and Version $1.1.9$ (Legacy, compatible with macOS $10.9+$).
  • Linux and ChromeOS: Version $2.0.0$ available via flathub.

Fundamental Characteristics of Psychological Science

• Psychological science is empirical: It is fundamentally based on measurement and observation, which results in data.

• Psychological science is objective: The procedures used are clear, transparent, and can be repeated by others.

• Psychological science is self-correcting: Theories and understandings are updated and revised as new data becomes available.

• Psychological science is progressive: Due to its self-correcting nature, the collective body of knowledge increases over time.

• Psychological science is tentative: Claims of having the "whole truth" are never made; instead, science maintains varying degrees of confidence about findings.

• Psychological science is parsimonious: When multiple explanations fit the data, the simplest explanation is preferred.

• Psychological science is concerned with theory: The ultimate goal is to develop theories regarding how things work and to establish what causes specific outcomes.

Research Questions in Health Research

• Differences: Research questions that investigate whether two or more groups differ on a specific outcome. An example is asking if a group receiving an experimental treatment shows a different outcome compared to a control group.

• Associations: Research questions that explore the relationship between two or more variables. An example is asking if increased ice cream sales are associated with higher rates of obesity.

Definition and Scope of Variables

• A variable is an event, object, or behavior that possesses at least two values (meaning it can vary) and is capable of being measured.

• Quantitative or qualitative attributes: Variables represent the quantity or quality of any attribute or phenomena that can be measured.

• Common examples of variables in health and psychology:

  • Physical metrics: Weight, height, blood pressure, blood glucose levels, cholesterol levels, and arm length.
  • Psychological metrics: Life satisfaction scores, personality scores, self-esteem, anxiety, intelligence, and physical attraction.
  • Demographic and social metrics: Socio-political attitudes, beliefs, mental health status, gender, age, ethnicity, and income.

Observing vs. Manipulating Variables

• Observing Variables: This involves recording information about variables without any manipulation. This is the domain of non-experimental research. Examples include:

  • Recording the number of students who consumed coffee before a lecture.
  • Recording how many students enjoyed a specific lecture.

• Manipulating Variables: This allows researchers to establish higher levels of certainty regarding cause-and-effect relationships. This is a primary benefit of experimental designs. An example includes:

  • Actively giving half of a class coffee before a lecture and then recording lecture enjoyment to see if the coffee caused a change.

Classification of Variables in Research

• Independent Variable (IV): The condition that is manipulated or selected by the experimenter to determine its specific effect on an outcome or on the dependent variable.

• Dependent Variable (DV): A measure of the participants' behavior that reflects the effect of the independent variable.

• Covariate (Extraneous Variables): Factors other than the independent variable that differ across participants and influence the dependent variable. These cannot be directly manipulated. Examples include genetics, age, and income. Covariates can be wanted or unwanted, but they must be measured and controlled for whenever possible, often through statistical adjustment.

• Confounding Variables: These occur when covariates have not been properly controlled. To control for unwanted covariates, researchers must measure them and employ adequate statistical techniques.

• Distinction between Covariates and Confounders: Covariates are independent variables that may or may not predict outcomes. While every confounder is a covariate, not every covariate is a confounder.

Correlations and Relationships Between Variables

• Correlation is a statistic that measures the linear relationship between Variable $x$ and Variable $y$.

• Use cases for correlation:

  • Examining variables that cannot be manipulated.
  • Examining variables where manipulation would be unethical.
  • Situations where experimental designs are inadequate.

• Directions of Correlation:

  • Positive Correlation: As one variable increases, the other increases. For example, alcohol consumption and aggressive behavior.
  • Negative Correlation: As one variable increases, the other decreases. For example, alcohol consumption and the quality of dance moves.
  • No Correlation: There is no discernable relationship. For example, brand of shoes and exam grades.

• Spurious Correlations: Correlations that appear significant but occur by chance rather than a causal link. The solution to spurious correlations is to collect more data.

  • Example 1: Piracy and Global Warming. A graph shows that as the number of active pirates decreased from $45,000$ to near zero, the Average Global Temperature increased from $14.4^{\circ}C$ to $16.0^{\circ}C$, with an $R^2 = 0.9839$.
  • Example 2: Ice cream consumption and murder rates. Data from May, June, and July often shows both ice cream sales and murders rising simultaneously, though one does not cause the other.

Causation and Experimental Manipulation

• Correlation does not equal causation. For example, bad smells and disease may correlate, but germs are the actual cause of disease.

• Observation alone often does not allow for inferences of cause and effect.

• To establish cause and effect, independent variables must be manipulated to observe the resulting changes in the dependent variable.

• Measurement: The process of assigning numbers to objects or events.

• Operationalization: The process of clearly defining a variable so it can be objectively measured. For example, quantifying "attractiveness" or "lecture enjoyment" for a study.

Experimental Designs

• Within-subjects (Individuals): A single subject undergoes both the experimental condition and the control condition.

• Between-subjects (Groups): Participants are divided into two distinct groups:

  • Experimental Group: Participants who receive the special treatment.
  • Control Group: Similar subjects who do not receive the treatment given to the experimental group.

NOIR: Levels of Measurement

• Nominal Scale:

  • Deals with relations, classes, and frequencies.
  • Attributes are only named; this is the weakest level of measurement.
  • Variables cannot be added, subtracted, multiplied, or divided.
  • Example: Smoker vs. Non-smoker.

• Ordinal Scale:

  • Numbers reflect rank order.
  • The distance between scores is not determined or meaningful.
  • Variables cannot be added, subtracted, multiplied, or divided.
  • Example: Likert scales where $1 = \text{strongly agree}$, $2 = \text{agree}$, $3 = \text{neutral}$, $4 = \text{disagree}$, and $5 = \text{strongly disagree}$.

• Interval Scale:

  • Numbers reflect both rank and a meaningful distance between scores.
  • These measurements are continuous.
  • Zero is arbitrary; there is no absolute zero.
  • Variables can be added or subtracted, but multiplication and division are not meaningful.
  • Example: Temperature.

• Ratio Scale:

  • Numbers reflect rank and meaningful distance between scores.
  • These measurements are continuous.
  • Zero is absolute, representing the complete absence of the attribute.
  • This is the highest level of specificity.
  • Examples: Income, height, weight, and the number of correct answers on a multiple-choice test.

• Hierarchical Specificity: Specificity increases from Nominal to Ratio. However, higher specificity does not mean a measure is "better." The research design and operational definitions determine the appropriate scale, which in turn dictates the data analysis and scope of conclusions.

Psychological Constructs

• Construct Definition: Psychological constructs are concepts (e.g., self-esteem, anxiety, intelligence, attraction) that are not directly observable and must be inferred from direct or indirect observations, including self-reports.

• Measurement of Constructs: To measure a concept like personality, it must be clearly defined beforehand.

Reliability and Validity

• Validity: Does the test measure what it is meant to measure? Accuracy.

  • Face Validity: Does the test look like it is measuring the right thing at a glance?
  • Internal Validity: The degree to which the change in the dependent variable is caused specifically by the independent variable.
  • External Validity: The extent to which the results of a study apply to the real world.
  • Construct Validity: Does the specific test effectively measure the concept being studied?

• Reliability: Does the test provide a repeatable, consistent measure? Precision.

  • Test-retest Reliability: Do the same individuals get similar results when taking the test twice?
  • Split-half Reliability: Does one half of the test produce similar results to the other half?

• Measurement Error:

  • Any instrument will overestimate as much as it underestimates.
  • The True Score is the score that would be obtained after infinite tests.
  • Equation for Obtained Score: $\text{Obtained score} = \text{True score} \pm \text{Measurement error}$

Recommended Reading List

• Source: Research Methods in Psychology by Howitt and Cramer (6th edition).

• Chapter 1: Role of Research in Psychology:

  • Section on Cause and Effect: Page $8$ onwards.
  • Section on Correlation: Page $11$ onwards.

• Chapter 3: Variables, Concepts and Measures:

  • Section on Variables: Page $45$ onwards.
  • Section on Scales of Measurement (NOIR): Page $53$ onwards.

• Chapter 16: Reliability and Validity:

  • Section on Reliability and Validity: Page $330$ onwards.