Research & stats w5 quant and survey

Quantitative designs

• experimental designs

• Surveys and questionnaires

Sample

• subset, small portion of population

• small sample advantages: time, cost, reliability

• Disadvantages of small: representativeness, how to sample correctly

Sample size

• amount of people in your sample

• Can be independent of population size. Depends on resources

• In quant, you can calculate what size of a sample needed for the study to be powerful enough to detect a difference

Effect size

SIZE OF EFFECT

• differences between groups

• How much did the intervention change behaviour

• How strong the association between two variables

• How time effects

Do notneed to learn the table

Correlation is signified with the letter…

R

Cohen’s d effect size

Small 0.2

Medium 0.5

Large 0.8

Power

• probability that you will correctly reject the null hypothesis

• 80% is generally seen as the minimum, threshold

Triangle!!

• power

• Sample size

• Effect size

larger sample size - get away with smaller effect size

Smaller sample size - must have larger effect size

Selection bias

• under/over-representation of subgroups. Lack of variation

• Often due to convenience sampling

Non-response bias

• members of sample refuse to participate

• We can’t study the characteristics of those who wouldn’t volunteer for research

System sampling

• pick every 4th person, etc. Systematic but random

Stratified random sampling

• a representative sample

• Have several strata (males, females) and randomly sample from that

Cluster sampling

• clusters in population. Ex: everyone from one neighborhood

Quantitative methods

• physiological (EEG, heart rate, blood pressure, skin conductance, pupillometry)

• Performance on task (reaction time, error rate etc)

• Questionnairres (numerical measurement or rating)

• Surveys (Capture experiences of phenomena)

Operationalisation in methods

• questionnaires - Rosenberg, Strengths and Difficulties Questionnaire

• Cogn tests - reaction time (to different images may show our sensitivity)

Reliability

• reliability - measures if we will get consistent scores

• Internal reliability - consistency within participant answers in one test. Ex: answering SUPER ANGRY but also not having angry feelings at all on same exam

• Test re-test

Validity

• measuring the concept correctly

• Face validity - questions clearly tapping into construct

• Discriminant val - ppl with anger issues and not shouldhave different scors

• Convergent val - diff ppl taking different test shsould get same score

• Divergent val - weak relationsips with diff concepts/tests are mesuring different things. Shouldn’t automatically same score on diff questions/tests

Psychometric

• thee measurement of psychological constructs

Cronback’s Alpha

• Way of quantifying if questions answer something similar

• Ex: 0.89

Questionaires pros/cons

• ask variety of questions

• Low cost

• Low time

• High generalisability

• Easy to achieve large sample size

Designing questionnaires

1. Likert scales

Likert scales

• numerical value to experiences

• Anchors - ex ”strongly disagree”

• What do the anchors mean? ”Somtimes” aka once a week? Once a day?

• Haing a midpoint can lead to central tendency bias

Visual analogue scale

• straight line with descriptive anchors

• Participants mark where their opinion is on the line

Open questions

• Write response in a box

• time consuming

• Require interpretation

• Nclude irrelevant information

Closed questions

• set boxes to choose options from

• Eliminate misinterpretation

• Quicker and easier to code

Reverse-coded items

• for most questions, a higher schore = greater anger

• But for ex opposite, kinder questions, flip the likert scale (behind the scenes) = ex: score high here = score low on anger

Informant-report

• when you answer questions bout someone else

• Ex: child, partner, student

Avoid confusing phrasing

• use regular day language

• Avoid double negatives

• Don’t ask 2 questions in 1. Ex: was your uni experience demanding and interesting? Yes/no

• If some questions are not applicable (as seen by the way they respond on earlier questions) make it simple for them to skip unnecessary/not applicable question

Minimize bias

• avoid phrases that could make certain responses more attractive to some groups

• Providing unbalanced response options

• Inlcuding questions with unnecessary emotions in question

• Being aware of the phrasing and how that impacts your validity

Minimise fence-sitting and floating

Fence-sitting - ppl who stay neutral through the whole test

Floaters - they don’t really know, but they provide an answer anyway

Pre-test your questions!!

1. Cognitive interviewing - think aloud responding - participants asked to verablise thoughts when answering

2. Interviewer probes - interviewer akss follow up questions about how the participant answerered the questions

Question order considerations

• context or priming effects our expectations for the next question/response option

• Add filters to your survey so ppl don’t have to answer irrelevant questions

Group administered

• being given questionnaire in a group, to fill out individually

Questionnaires in the post

• Get in post, fill out, put stamp and send back

• Rely on participants returning questionnaires. Must pay postage too :(

Electronic surveys

• self-administered, structured, online

• Low cost, easier to randomize question order

• Google forms, Pavlovia, Qualtrics, etc

Phone interviews

• not as useful, but good way of getting responses

• Structured interview

• High time commitment needed from researcher

• Not as many people pick up, either busy or sussy numberrrrr

In-person interviews

• go through the questions one by one

• Structured or unstrutured

• High response rate

• Longer, more complex questionnaires

• Controlled setting

• Not time efficient

• Not cost efficient

• Experimenter’s presence may impact what participant shares abt themselves