Scientific Research Methods

Limitation of Human Thinking:

• BIAS!!!

  • fact check with one (or multiple) empirical research studies

  • Multiple Types

    • mental heuristics

    • availability heuristics: you only use the information you readily have in your brain

      • recency, frequency, extreme, negative, vivid

        • ex. news just read, traumatic experiences

    • confirmation bias: the tendency to only search for evidence that fits beliefs while ignoring evidence not fitting beliefs

      • ex. MS ted talk; searched MS in wheel chair instead of just MS information

      • counteract: look for information that goes against your belief

    • hindsight bias: belief made when you already know something after the fact

      • people tend to overestimate the predictability of prior events

        • ex. after test, “i knew she was gonna ask this on the test, why did i not study it?”

• fast vs slow thinking

  • fast: common, easy and automatic, emotional, more prone to bias and errors

  • slow: rare, hard and deliberate, rational, reliable

Scientific Research/Process

• science is a process of thinking and investigating

  • the process is deliberate, systematic, effortful

  • working with empirical evidence

  • designed to avoid bias as much as possible

• the conclusions are tentative and can be interpreted differently

  • important to be open minded and consider different perspectives and contexts

  • important to verify information and be aware of limitations of our knowledge

• science does not necessarily prove the truth

• systemic empiricism:

  • systematic = follows organized method/plan

  • empiricism = relies on observable evidence

    • ignoring either leaves more room for bias

• public verification: information/data ready for public verification,

  1. all ideas: only small part of idea can be shared with the public

  2. initial research process: scientific training and self judgment helps researcher focus on important and viable ideas

     • ask research question

  3. research programs: peer review minimizes

  4. published research: published research should provide sufficient details for others to replicate the work

  5. established knowledge:

• solvable problem: information added through research

  • a solvable problem is falsifiable (can be proved wrong with observable evidence)

    • ex. aliens exist?

      • assume no = scientific method

      • assume yes = confirmation bias

    • ex. a personality test shows you have some weaknesses

      • not solvable/falsifiable: what evidence would prove it?

    • ex. if you don’t exercise, you will eventually develop health problems

      • cannot be falsified, you could measure to the end of life and death will be “health problems”

      • specify for falsifiability

    • ex. people’s unfulfilled unconscious desires cause mental health problems

      • can’t falsify, how do you evidence unconscious desires?

  • assumption with no information: assume nothing without evidence

    • can’t prove something that doesn’t exist

    • leads to confirmation bias

  • The Research Process:

    1. develop a good (specific and measurable) question

    2. develop a hypothesis (priori hypothesis formed based on theory to avoid hindsight bias)

    3. operationalize the variable

       • what does it “mean”, what kind of data should i observe?

       • ex. “i want to study ai” but what is ai, how do you measure it? same for stress

         • self report? not always as reliable

    4. design study and collect data

    5. analyze the data

    6. draw conclusion (tie into research question and hypothesis)

       • blue = method, purple = concept

       • share/publish result