Fundamentals of social science research - Week 4

Variables + measures + validity + reliability

Independent variable

• Variable that changes

  • May assign ppt

  • Manipulation of condition by researcher

Dependent variable

Outcome of variable

• What ppt does back

• Measure behaviour that reflects IV effect

Different types of variables

• Categorical: A selection

• Quantative: Varies in amount

• Continuous: Number can be infinite

  • Age, speed, height etc

• Discrete: Specific amount

  • Number of students

Confounding variables

External variables that have an effect on the IV

Different types of measurement scales

Nominal, ordinal, interval, ratio

Nominal

Numbers

• Divide objects/events into categories based on similarities/differences

• No numerical value

• If there is a number, it is ARBITRARY

Ordinal

Order

• Rank/Reflect degree of magnitude, interval may not be equal

Interval

Equal gaps

• Differences between numbers are meaningful

• Intervals between numbers are equal

• e.g., temperature in degrees Celsius: difference between 20°Cand 10°C is 10 degrees. Difference between 40°C and 30°C is also 10 degrees

Ratio

Equal gaps with true zero = Zero is meaningful

• Like internal but with true zero point

• Values can be compared as ratios

• e.g., weight: 0 kg = absence of weight. 20kg is twice as heavy as 10 kg

Reliability

Consistency in results

Types of reliability

Test re-test, inter-rater, internal consistency, split half reliability

Test re-test

Measures reliability of test

e.g. If someone is given a grammar test today and the week after, they should score relatively similar

Inter-rater

Two people should score the same script

e.g. if two people are marking a test, they should mark that person with around the same score

Internal consistency

a measure of how well different items on a test, survey, or scale measure the same underlying construct or concept

Split half reliability

items on the tests are divided into 2 sets as if they were 2 separate tests -> correlated to see how closely the various indivual scores agree on the 2 halves -> good tests will have split-half correlation

Different types of validity

Construct, face, content, criterion

Construct validity

Are you measuring what you want to measure

Face validity

Does the test appear to measure what it’s meant to measure

E.g. language: name a pencil and watch

Content validity

Does it comprehensively measure all aspects represented by the theoretical concept

• E.g. cognitive function: memory/language/attention

Criterion validity

• Does it correlate with other measures of the same construct

  • E.g. does the test correlate with other objects

Threats against internal validity

• Can you claim causality?

• Temporal precedence: Which one affects the other?

• History: Events that happened outside the study

• Maturation: Passage of time between the experiment

• Testing: Ppt learning from repeated testing

• Regression towards the mean: Extreme scores on the first test tend to approach the mean on subsequent tests

• Selection: Bias in group assignment

• Mortality: Ppt drop out due to lack interest, death, moving away

Threats to construct validity

• Do the results support the theory behind the research?

• Would another theory predict the same results?

Threats to external validity

How well can you generalise the findings to other populations or situations?

Threat to statistical validity:

Was the relationship between the IV and DV spurious (fake/false)?

Control in experiments?

Any means to rule out possible threats to the validity of the research to achieve equivalent groups. Done through control groups.

Alternative way of control?

Control without active/passive control group —> Do an alternative of the experiment

Between participant design

Each ppt only experiences one condition

Between participants matching

where researchers pair up similar subjects and assign one to the treatment group and the other to the control group

Within participant design

Each participant experiences every condition