Quantitative Research Methods and the Scientific Method in Psychology

Central Questions of Psychological Research

• The study of psychology involves addressing several fundamental questions regarding the validity and acquisition of knowledge about human beings:

  • How do we know when we know something about people?

  • What methods or combinations of methods provide the most conclusive results or insights?

  • What is gained and what is lost when a psychological phenomenon is 'domesticated' by bringing it into a laboratory setting?

  • How can a deeper understanding of phenomena be gained?

  • How do we know if the right questions were asked in the first place?

Understanding Quantitative Research in Psychology

• Definition of Quantitative Research: Typically involves using experiments or surveys to gather data that can be statistically analysed to test particular hypotheses.

• The Quantitative Process:

  1. Carry out an experiment or survey.

  2. Tabulate the collected data.

  3. Analyse the data using standard statistical packages and protocols.

  4. Draw conclusions about predetermined hypotheses.

• Characteristics of Quantitative Research:

  • Based on large sample sizes representative of the population.

  • High reliability due to the ability to replicate or repeat research.

  • Objective analysis using standardised techniques.

  • Deductive and objective approach.

  • Conclusions are drawn from scientifically controlled testing.

• Major Goals of Empirical Research:

  • Description: Telling 'what occurred'. Psychologists name and classify behaviours and mental processes through careful observation.

  • Prediction: Identifying when and under what conditions a future behaviour or mental process is likely to occur.

  • Explanation/Understanding: Telling 'why' a behaviour or mental process occurred. Research questions aim to learn more about a topic and answer the cause of a phenomenon.

Characteristics of Scientific Psychological Research

Scientific research shares several common attributes to ensure dependable results and minimise error:

• Theoretical Framework: A systematic way of organising and explaining observations.

• Standardised Procedure: A procedure that remains the same for all participants, except where variation is introduced specifically to test a hypothesis.

• Generalisability: The use of a sample that is representative of the broader population.

• Objective Measurement: Use of measures that are reliable (consistent results) and valid (assessing what they purport to assess).

The Scientific Method

According to Sanderson and Huffman (2019), the scientific method follows a progression of logical steps:

1. Observation and Literature Review: Identifying a question of interest and reading previously published work in major scientific journals.

2. Testable Hypothesis: Developing a specific prediction about how one factor (variable) relates to another. Variables must be operationally defined—stated precisely in measurable terms.

3. Research Design: Choosing the best design to test the hypothesis (e.g., experimental, descriptive, or correlational).

4. Data Collection and Analysis: Performing statistical analyses to determine if findings are statistically significant and if the original hypothesis is supported or rejected.

5. Publication: Submitting a written study to a peer-reviewed journal for critical evaluation by other scientists.

6. Theory Development: Proposing new theories or revising existing ones based on published results, leading to new hypotheses.

Theoretical Frameworks and Hypotheses

• Theory: A systematic way of organising and explaining observations, including propositions about relationships among phenomena.

  • Example: A theory might state that pessimism promotes poor physical health because pessimists neglect self-care and pessimism keeps the body in a constant state of alarm.

• Hypothesis: A tentative belief about the relationship between two or more variables, predicting findings if a theory is correct.

• Variable: Any phenomenon that can differ or vary from one situation or person to another (e.g., IQ scores of $115$ or $125$).

• Continuous Variable: A variable that can be placed on a continuum, such as degree of optimism, intelligence, shyness, or rate of recovery.

• Categorical Variable: Comprised of groupings or categories that cannot easily be placed on a continuum, such as species, a person's home state (e.g., Queensland), or whether a person has had a heart attack.

Standardised Procedures

• Definition: Exposing participants to as similar procedures as possible to ensure that results reflect the variables being tested rather than procedural differences.

• Case Study: Emotional Disclosure and Academic Performance (Lumley and Provenzano, 2003):

  • Emotional Disclosure Group: Wrote for $15-20\,\text{minutes}$ on $4$ consecutive days about deepest thoughts regarding a traumatic experience.

  • Control Group: Wrote about time management goals for the same duration, with instructions not to mention emotions.

  • Results: The emotional disclosure group had significantly better Grade Point Averages (GPAs) the following semester.

  • Procedural Note: Standardisation (writing for exactly $15-20\,\text{minutes}$) ensured the result was due to content, not the quantity of writing.

Generalisability and Sampling

• Population: The larger group of interest to which research findings should apply (e.g., all humans or preschool children with working mothers).

• Sample: A subgroup of the population likely to be representative of the whole.

• Participants/Subjects: Individuals who provide informed consent and participate in the study.

• Generalisability: The applicability of findings to the entire population. This depends on having a representative sample.

• Sampling Bias: Occurs when certain elements have a greater or lesser chance of being selected, leading to over-representation or under-representation.

Internal and External Validity

• Internal Validity: The validity of the research design itself; methods must convincingly test the hypothesis without fatal flaws (like non-standardised aspects).

• External Validity: The extent to which findings can be generalised to situations outside the laboratory.

  • Replication: Repeating results using different data collection procedures is crucial for establishing external validity.

• The Balance: Tighter laboratory control increases internal validity but may decrease external validity by making the situation less like real life.

Objective Measurement

To ensure objectivity, researchers must quantify or categorise variables.

• Measure: A concrete way of assessing a variable, bringing abstract concepts 'down to earth'.

• Case Study: Popularity in Primary School Boys (Rodkin et al., 2000):

  • Researchers challenged the idea that all popular kids are 'model citizens'.

  • Method: Teachers used an $18$-item questionnaire to rate boys on a numerical scale of $1-7$ ($1 = \text{not true}$, $7 = \text{very true}$).

  • Average Measure: Popularity was assessed by averaging ratings on 'popular with girls', 'popular with boys', and 'lots of friends'.

  • Findings: Two types of popular boys identified: 'Model Citizens' (academic, friendly, athletic) and 'Aggressive' (athletic, good-looking, but aggressive).

Reliability of Measures

Reliability refers to the ability to produce consistent results, regardless of random factors like sleep or who coded the data.

• Retest Reliability: The tendency of a test to yield similar scores for the same individual over time.

• Internal Consistency: Different ways of asking the same question yield similar results.

• Interrater Reliability: Different interviewers or observers give similar scores to the same individual.

• Developing Reliability: Complex variables (like optimism in diaries) require detailed coding manuals to ensure raters are 'calibrated'.

• Plumber Analogy for Reliability:

  • Retest: Showing up when they say they will on different occasions.

  • Internal Consistency: Fixing a toilet as efficiently as a sink.

  • Interrater: Customers agreeing on the quality of work.

Validity of Measures

• Definition: The ability of a measure to assess the variable it is intended to assess (e.g., IQ tests measuring intelligence).

• Validation Research: Demonstrating that a measure relates to an objective criterion or other already-validated measures.

• Predictive Power: A valid measure should predict other variables to which it is theoretically related (e.g., teacher reports of popularity predicting school dropout rates $8$ years later).

Test Bias and Minority Groups

• Criteria for Test Bias:

  1. Systematic differences in mean scores between different groups.

  2. Test scores making incorrect predictions in real life.

• Case Study: Māori Job Applicants and Cognitive Tests (Guenole et al., 2003):

  • Māori participants scored $0.55$ and $1.79$ standard deviations lower than European counterparts on two cognitive tests.

  • Diagnosis of Bias: The tests were biased because they assumed prior knowledge of business terminology.

  • Contextual Factors: Differences are often due to social disadvantage and prejudice rather than a lack of ability (Lilienfeld et al., 2010).

Multiple Measures

• No psychological measure is perfect; every measure contains a degree of error or discrepancy.

• A measure that is accurate $80\%$ of the time is also inaccurate $20\%$ of the time.

• The 'Safety Net': Using multiple measures to assess the same variable helps catch errors and provides a more accurate assessment.

Comparison of Quantitative Research Methods

Questions & Discussion

• APAC Foundational Competency (1.1.xii.): Focuses on 'research methods and statistics'.

• Application Task: Consider why and how psychologists apply research methods and statistics to understand human behaviour and mental processes.

• Self-Reflection: Reflect on personal reasons for studying psychology and theorise why others do. How could different methodologies (quantitative and qualitative) test these theories?

• Connections Question: To what extent is pessimism a cause or consequence of depression? How might a researcher test if a pessimistic style predisposes people to later depression?

• Generalisability Question: Can results from a US-based emotional disclosure study be generalised to Australian or New Zealand students, especially in cultures that discourage emotional expression?

• IQ and Talent: If an IQ test predicts school success but not artistic genius (e.g., Taika Waititi), is it a valid measure of intelligence? What assumptions underlie paper-and-pencil IQ tests?