Psychological Research Methods, Statistics, and Ethics

The Need for Psychological Science and Critical Thinking

• Common sense and intuition often fail to provide reliable answers about human behavior and thought processes. Science-based answers are far more valid because they rely on empirical observation, systematic testing, and rigorous analysis.

• Common sense statements often sound true due to the repetition effect: repeating a statement makes it easier to remember and creates an illusion of truth, even when scientific evidence refutes it.

• Evaluation of Common Intuitions:

  • Vitamin C prevents the common cold: False.

  • We only use $10\%$ of our brain: False.

  • The more we are around someone, the more they get on our nerves: False.

• Three Primary Roadblocks to Critical Thinking:

  1. Hindsight Bias:

  • Definition: The tendency to believe, after learning an outcome, that one would have foreseen it (commonly known as the "I-knew-it-all-along" phenomenon).

  • Philosophical foundation: Soren Kierkegaard noted, "Life is lived forwards, but understood backwards."

  • Historical / Real-World Example:

    • The $2010$ Deepwater Horizon BP oil spill: Employees took shortcuts and ignored warning signs without intending to cause an environmental catastrophe. After the spill occurred, the foolishness of those decisions became painfully obvious through $20/20$ hindsight.        

      

  • Social Examples:

    • Couple breakups: After a couple breaks up, friends state, "They were never a good match." However, if the same couple had married, friends would claim, "They are perfect together."

    • Sports events: Following a lost Friday night football game, fans complain, "That stupid move by the coach lost us the championship!" Conversely, following a win, fans declare, "That gutsy move by the coach was a great call!"

  • Proverbial Contradictions:

    • When presented with "Separation weakens romantic attraction" ("Out of sight, out of mind"), subjects find it unsurprising and true.

    • When presented with "Separation strengthens romantic attraction" ("Absence makes the heart grow fonder"), subjects also find it unsurprising and true.

  1. Overconfidence:

  • Definition: The tendency to think we know more than we actually do, exaggerating the correctness or accuracy of our beliefs and predictions.

  • Anagram Experiment Demonstration:

    • Unscramble anagrams: WREAT ($W-A-T-E-R$), ETRYN ($E-N-T-R-Y$), GRABE ($B-A-R-G-E$).

    • When problem solvers attempt the task without answers provided, the average solution time is $3\text{ minutes}$.

    • When given the anagrams alongside the solutions, subjects estimate that it would take them only $10\text{ seconds}$ to solve them.

  • Historical Overconfidence Predictions:

    • British expert group evaluating the telephone: "The telephone may be appropriate for our American cousins, but not here, because we have an adequate supply of messenger boys."

    • Popular Mechanics ($1949$): "Computers in the future may weigh no more than $1.5\text{ tons}$."

  • World Event Expert Predictions:

    • Experts expressed an average of $80\%$ confidence regarding $27,000$ world event outcomes. Their predictions were correct only $40\%$ of the time.

  1. Perceiving Order in Random Events:

  • Definition: In an eager attempt to make sense of an unpredictable world, humans are naturally prone to perceive patterns, structure, or sequences where none exist.

  • Coin Flip Probability Example:

    • When flipping a fair coin $6$ times, people typically believe that a sequence such as $\text{HTTHTH}$ is more likely than $\text{HHHTTT}$ or $\text{HHHHHH}$.

    • Science demonstrates that all three sequences are equally likely (or equally unlikely), with a probability of $(1/2)^6 = \frac{1}{64}$.

  • Randomness and Sampling:

    • Random sequences frequently do not look random to human perception.

    • Financial fraud experts identify embezzlers specifically because embezzlers try too hard to make their illicit withdrawals look random.

    • Statisticians Persi Diaconis and Frederick Mosteller ($1989$) summarized this phenomenon: "With a large enough sample, any outrageous thing is likely to happen."

The Scientific Method and Research Design

• Definition: Psychologists utilize the scientific method to support a scientific attitude. It is a self-correcting process that uses systematic observation and statistical analysis to evaluate ideas.

• Core Premise: Empirical evidence supersedes intuition ("The rat is always right").

• The Scientific Method Cycle:

  1. Theory: An explanation using an integrated set of principles that organizes observations and predicts behaviors or events.

  • Example: Observing that well-rested students retain information better leads to the theory that sleep boosts memory.

  1. Hypothesis: A testable prediction, often implied by a theory.

  • Example: When sleep-deprived, people remember less information from the day before.

  1. Research and Observation: Systematically testing the hypothesis under controlled or measurable conditions.

  • Example: Administering study material to subjects before either an ample night's sleep or a shortened night's sleep, then testing their memory.

  1. Confirmation, Rejection, or Revision: The outcome of research leads scientists to confirm, reject, or revise the original theory.   

     

• Operational Definitions:

  • Definition: A carefully worded, explicit statement of the exact procedures (operations) used in a research study.

  • Function: Quantifies abstract variables so they can be measured precisely.

  • Example (Human Intelligence): Defined operationally as the numerical score achieved on a standardized intelligence test.

  • Example (Sleep Study Operationalization):

  • "Sleep loss / Sleep deprived": Operationally defined as getting $2\text{ fewer hours}$ than the subject's baseline/normal sleep duration.

  • "Remember less": Operationally defined as comparing the total number of words correctly recalled on a standardized memory list following a normal night of sleep versus a shortened night of sleep.

• Replication:

  • Definition: Repeating the essence of a research study, usually with different participants in different situations, to see whether the basic finding extends to other participants and circumstances.

  • Operational definitions make precise replication possible. In scientific research, replication is confirmation.

• Three Main Research Categories to Test Hypotheses:

  1. Descriptive Methods: Describe behaviors, thoughts, or attributes through case studies, naturalistic observations, or surveys.

  2. Correlational Methods: Associate different factors or variables to determine how well one predicts another.

  3. Experimental Methods: Manipulate factors to discover cause-and-effect relationships.

Descriptive Research Methods

• Purpose: To observe and describe behavior without manipulating variables.

• Three Types of Descriptive Methods:

  1. Case Studies:

  • Definition: A descriptive technique in which one individual or one small group is studied in depth in the hope of revealing universal principles.

  • Subject Scope:

    • Single Individuals: Patient H.M. (memory loss), Little Hans (psychoanalytic phobia study), Genie (the feral child studying language acquisition).

    • Single Groups: University of Tennessee women's basketball team; prison inmates participating in a group therapy study.

  • Strengths:

    • Enables detailed examination of rare, atypical, or unusual behaviors.

    • Provides a rich, extensive amount of qualitative data.

    • Generates theoretical hypotheses and suggests directions for future research.

  • Limitations:

    • Atypical individual cases can be misleading or unrepresentative.

    • Findings cannot be generalized to the broader population.

    • Cannot establish cause-and-effect relationships.

  1. Naturalistic Observation:

  • Definition: A descriptive technique of observing and recording behavior in naturally occurring situations without trying to manipulate or control the situation.

  • Examples:

    • Jane Goodall observing chimpanzee social structures in the African jungle.        

      

    • Big Data Analysis: Tracking positive and negative emotion words in $504\text{ million}$ Twitter messages across $84\text{ countries}$.        

      

    • Twitter analysis showed that positive words in tweets peak late Saturday night and reach their lowest level on Tuesday afternoon.

  • Strengths:

    • Participants act naturally because they are in real-world environments outside a laboratory setting.

    • Data collection is unobtrusive and does not interfere with the subject's behavior.

  • Limitations:

    • Independent variables cannot be controlled or isolated.

    • Cannot determine cause-and-effect relationships.

    • Researcher observations may be subjective or influenced by observer bias.

  1. Surveys and Interviews:

  • Definition: A descriptive technique for obtaining the self-reported attitudes or behaviors of a particular group, typically by questioning a representative, random sample.

  • Strengths:

    • Quick and efficient method to assess public opinion, beliefs, or behaviors.

    • Capable of gathering data from a large number of cases.

  • Limitations:

    • Response Bias: Participants may answer dishonestly or give socially desirable answers.

    • Wording Effects: Subtle changes in word choice can dramatically alter participant responses.

      • Comparative Examples:

      • "Aid to the needy" receives significantly higher public support than "welfare".

      • "Gun safety laws" receives different approval ratings than "gun control laws".

      • "Revenue enhancers" receives more support than "taxes".

    • Sampling Bias: Conducting a survey on an unrepresentative group skews results (e.g., claiming $94\%$ of people support NASA funding without disclosing that the sample was taken near a space research facility).

    • Cannot determine cause and effect.

• Sampling Terminology and Demographics:

  • Population: The entire target group being studied, from which samples may be drawn.

  • Representative Sample: A sample that accurately reflects the demographic distribution of the overall target population.

  • Example: If a school's population consists of $25\%$ Hispanic, $30\%$ Asian, $35\%$ African American, and $10\%$ Caucasian students, and $50\%$ male / $50\%$ female, a representative sample must match those identical demographic percentages.

  • Random Sample: A sample in which every single individual in the population has an equal chance of being selected for inclusion.

Correlational Research Methods

• Definition: Correlation is a measure of the extent to which two factors vary together, and thus how well either factor predicts the other.

• Examples of Correlated Variables:

  • Cigarette smoking behavior correlates with lung cancer rates.

  • Study skills correlate with Grade Point Average (GPA).

• Positive Correlation vs. Negative Correlation:

  • Positive Correlation:

  • Definition: Two sets of data tend to rise together or fall together (move in the same direction).

  • Scatterplot Slope: Rises UP from left to right.

  • Example: College graduation and income levels. Individuals who complete college (rise) tend to earn higher incomes (rise).     

    

  • Negative Correlation:

  • Definition: One set of data rises while the other set falls (move in opposite directions).

  • Scatterplot Slope: Rises DOWN from left to right.

  • Example: Age and amount of nightly sleep. As age increases (rise), nightly sleep duration decreases (fall).     

    

  • No Relationship:

  • Definition: Variables increase or decrease randomly; no predictive pattern exists.

  • Scatterplot Representation: Dispersed dots with no visible line or trend slope ($r = 0.00$).     

    

• Scatterplots:

  • Definition: A graphed cluster of dots, each representing the values of two variables.

  • Interpretation:

  • Slope: Indicates the positive or negative direction of the relationship.

  • Scatter: The concentration or spread of dots indicates the strength of the correlation (less scatter = stronger correlation).

  • Axis Quantification: Values on both the x-axis and y-axis must be numeric and quantified.

• Correlation Coefficient ($r$):

  • Definition: A statistical index measuring the direction and numerical strength of the relationship between two variables.

  • Scale Range: Ranges from $r = -1.00$ to $r = +1.00$.

  • Positive Correlation Range: $r = +0.01$ to $r = +1.00$ (Perfect positive correlation = $r = +1.00$).

  • Negative Correlation Range: $r = -0.01$ to $r = -1.00$ (Perfect negative correlation = $r = -1.00$).

  • Key Rule for Interpretation:

  • The plus ($+$) or minus ($-$-) sign indicates **direction** only, not value or magnitude.\n - The absolute numerical value (distance from zero) indicates the **strength** of the correlation. A correlation of r = -0.85$is stronger than a correlation of$r = +0.50.\n\n- **Illusory Correlations**:\n - Definition: Perceiving a relationship between variables where none exists, or perceiving a stronger-than-actual relationship.\n - Example: A sports fan wears a specific blue jersey on the day her team wins a game. She subsequently insists on wearing that blue jersey every game day, believing it affects the outcome, even though the jersey has no causal impact on the score.\n - Spurious Correlation Example: Data compiled by tylervigen.com demonstrates a high correlation between the number of people who drowned by falling into a swimming pool and the number of films actor Nicolas Cage appeared in between 1999$and$2009.\n  \n\n- Prediction vs. Causation:\n - Correlations provide predictive value, but **correlation does not prove causation**.\n - Self-Esteem and Depression Example:\n - Self-esteem correlates negatively with depression (lower self-esteem predicts a higher risk of depression).\n - Three Possible Interpretations:\n 1. Low self-esteem causes depression.\n 2. Depression causes low self-esteem.\n 3. A third variable (e.g., distressing life events, biological predisposition) causes both low self-esteem and depression.\n    \n - Mental Illness and Smoking Example:\n - Mental illness correlates positively with smoking.\n - Possible causal pathways: Smoking leads to mental illness; mental illness leads to smoking; or a third variable (such as a stressful home environment) triggers both smoking and mental illness.\n    \n - Male Marriage Duration and Hair Loss Example:\n - Length of marriage correlates positively with hair loss in men. This does not mean marriage causes male pattern baldness or that balding men make better husbands; aging is the underlying confounding factor.\n\n\n# Experimental Research Methods\n\n- Purpose: Experiments enable researchers to isolate cause-and-effect relationships by:\n 1. Manipulating the factors of interest (independent variables).\n 2. Holding constant (controlling) other extraneous factors.\n\n- Comparative Study Case: Breastfeeding and Intelligence (Kramer et al., 2008):\n - Research Question: Does breastfeeding lead to smarter children?\n - Study Design:\n - **Experimental Group**: Pediatricians promoted breastfeeding to nursing mothers in their care. At 3\text{ months}$of age,$43\% of the babies were being breastfed.\n - **Control Group**: Pediatricians provided standard care without promoting breastfeeding. At 3\text{ months}$of age,$6\% of the babies were being breastfed.\n    \n - Experimental Results:\n - At age 6$, when nearly$14,000$of the children were restudied, children in the breastfeeding-promotion experimental group scored an average of$6\text{ points}$$ higher on intelligence tests than those in the control group.

• Key Experimental Concepts:

  • Experimental Group: The participants in an experiment who receive the treatment (the manipulation of the independent variable).

  • Control Group: The participants in an experiment who do not receive the experimental treatment; used as a baseline for comparison.

  • Random Sampling vs. Random Assignment:

  • Random Sampling:

    • Definition: Selecting a representative sample from a population by chance.

    • Purpose: Allows experimental or study findings to be generalized to the broader population as a whole.

  • Random Assignment:

    • Definition: Assigning participants to experimental and control groups by chance.

    • Purpose: Minimizes pre-existing individual differences between groups, controlling for confounding variables.

  • Blind Procedures and Control of Bias:

  • Single-Blind Procedure:

    • Definition: Research participants are uninformed (blind) regarding whether they are receiving the treatment or a placebo.

    • Controls for: Subject response bias and the placebo effect.

  • Double-Blind Procedure:

    • Definition: Both the research participants and the experiment staff/researchers are uninformed about which group received the treatment or placebo.

    • Controls for: Experimenter bias, subject response bias, and the placebo effect.

  • Placebo and Placebo Effect:

    • Placebo: An inert treatment, such as a sugar pill containing no active pharmacological ingredient.

    • Placebo Effect: Experimental results caused by participant expectations or assumptions alone.

  • Experimental Variables:

  • Independent Variable (IV):

    • Definition: The factor that is manipulated by the experimenter; the variable whose effect is being investigated.

    • Administration: Given strictly to the experimental group.

  • Dependent Variable (DV):

    • Definition: The outcome factor that is measured; the variable that may change in response to manipulations of the independent variable.

    • Administration: Measured in both the experimental group and the control group.

  • Confounding Variable:

    • Definition: A factor other than the independent variable that might influence a study's results.

    • Criterion: A variable is only confounding if it is uniquely present in one group and not the other. It cannot be confounding if it occurs to the same degree in both groups.

    • Examples of Confounding Variables: Age, baseline intelligence level, ethnicity, biological sex, political beliefs.

    • Prevention Strategy: Random assignment equalizes pre-existing individual differences across groups.

  • Experimental Validity:

  • Definition: The extent to which a test or experiment measures or predicts what it is intended to measure or predict.

  • Example: Passing a practical driving test is considered a valid measure of an individual's actual safe driving ability.

• Visual Aside:

  • Reggie the dog (demonstrating measurement scales).   

    

Ethical Guidelines in Psychological Research

• Animal Research Ethical Guidelines:

  • British Psychological Society Guidelines: Requires housing research animals under reasonably natural living conditions and providing social companions for social species.

  • American Psychological Association (APA) Guidelines: Mandates that researchers provide "humane care and healthful conditions" and conduct testing that "minimizes discomfort".

  • Institutional Governance: University Institutional Animal Care and Use Committees (IACUC) screen research proposals, and animal laboratories are subject to federal regulation and regular inspections.

• Human Research Ethical Guidelines:

  1. Informed Consent:

  • Definition: Informing prospective participants about all aspects of the research that might reasonably be expected to influence their willingness to participate (e.g., potential risks, discomforts, or seizure-inducing stimuli) prior to securing agreement.

  • Comparison:

    • Simple Consent: "Are you willing to participate in this experiment?"

    • Informed Consent: "This experiment involves exposure to graphic images that may be disturbing and random bursts of light that have been known to induce seizures. Are you willing to participate in this experiment?"

  1. Protection from Harm:

  • Researchers must safeguard participants from unnecessary physical pain and psychological distress.

  1. Confidentiality:

  • Individual participant identity and personal data must be kept strictly private and anonymous.

  1. Right to Withdraw:

  • Participants must be allowed to leave the experiment at any time without penalty or coercion.

  1. Debriefing:

  • Definition: The post-experimental explanation of a study, including its purpose, results, and any deceptions used, given to participants upon completion.

  • Deception Policy: Temporary deception is permitted only when revealing the true research purpose beforehand would invalidate the experimental results. Full debriefing is required immediately at the study's conclusion.

Review Questions and Analytical Exercises

• Practice Exercises: Operationalizing Hypotheses

  • Theory 1: "Music helps plants grow."

  • Hypothesis Formulation: Exposed plants will grow taller than unexposed plants.

  • Terms to Operationalize: Genre/volume of music, duration of daily exposure, plant species, measure of growth (height in centimeters).

  • Theory 2: "Ginger tea helps reduce hyperactivity in teens."

  • Terms to Operationalize:

    • Tea Administration: Exact fluid volume drank, ingredient concentrations, daily frequency of consumption.

    • Target Population: Exact age range of teens, biological sex, clinical diagnosis status (e.g., diagnosed ADHD vs. neurotypical).

    • Hyperactivity Measurement: Numerical score on a standardized behavioral rating scale or observed motor movements per hour.

  • Theory 3: "Eating junk food causes weight gain."

  • Terms to Operationalize: Caloric intake of specified high-fat/high-sugar foods, duration of diet, weight measurement (kilograms gained over a fixed time frame).

• Sample Conceptual Review Questions:

  • Question 1: After a student council election, a friend claims he could have predicted who would win. What psychological phenomenon does this illustrate?

  • Options: A. Common sense | B. Critical thinking | C. Hindsight bias | D. Overconfidence | E. Perceiving order in random events.

  • Correct Answer: C. Hindsight bias.

  • Question 2: The tendency to exaggerate the correctness or accuracy of our beliefs and predictions is called:

  • Options: A. Hindsight bias | B. Overconfidence | C. Critical thinking | D. Skepticism | E. Reliability.

  • Correct Answer: B. Overconfidence.

  • Question 3: A local basketball team has won three championships in a row and is on a winning streak entering the final game. Explain how fans might use hindsight bias, overconfidence, and perceiving patterns in random events to explain a loss in the final game:

  • Hindsight Bias Explanation: After the team loses, fans claim they knew all along that the winning streak would end because the team was overconfident or due to a specific coaching call.

  • Overconfidence Explanation: Before the game, fans assumed victory was guaranteed based on the streak, ignoring tactical match-ups.

  • Perceiving Patterns Explanation: Fans interpret random occurrences during the game (e.g., missed free throws or referee calls) as part of a deterministic pattern or "curse" dictating the outcome.