HSCI 207 quiz 2

sos

Critiques of quantitative research (Human Behaviour vs. Natural Science)

• Quantitative research treats human beings as if they were natural phenomena such like atoms or chemicals

• Quantitative researchers argue that since humans are part of nature, scientific methods can still help understand human behaviour

Critiques of quantitative research (Artificial Precision and Measurement issues)

• Quantitative measurements creates an illusion of precision and accuracy

• This can lead to measurement validity issues, where numbers fail to reflect real meanings or contexts

Critiques of quantitative research (Disconnection from real life)

• Use of controlled instruments and procedures makes research too detached from everyday experiences

• Which reduces external validity

Critiques of quantitative research (Neglect of subjective meanings)

Quantitative research focuses on relationship between variables while ignoring how people define and interpret their own experiences

Critiques of qualitative research (disregard for participants perspectives)

• Even when quantitative results are valid, they may not reflect what participants actually think or feel

Critiques of qualitative research (Objectivist ontology)

• Quantitative research assumes that social reality exists independently of individuals as if society were fixed and objective

• Overlooks how social reality can be constructed, fluid and shaped by human interactions

4 Goals of Quantitative research 

1. Measurement => Understand data

2. Establish causality => cause and effect

3. Generalization of findings => if able to use findings outside of experiment

4. Replication => being able to redo the experiment on different participants and get the same results

Nominal level of measurement

• Qualitative 

• Least precise 

• No mathematical interpretation

Ordinal level of measurement 

• Characteristics can be put into categories and be ordered in an purposeful way

• Rank ordered according to amount of characteristic the object has

• Mutually exclusive 

• Distances between variables are not equal across the range

Interval level of measurement

• No rank orders

• Actual values between has meaning (1-2 is the same as 4-5)

• Numbers have value but no true zero point

Ratio levels of measurement

Have all characteristics of nominal, ordinal, interval levels and have an absolute zero point which represents absence

Indicators

• Something employed to measure a concept

• Can be direct or indirect measures of concept

• Tells us there might be a link and how strong it is

Reliability

• Consistency or stability of a measurement

1. Stability over time

2. Internal reliability 

3. Inter-observer consistency

Stability over time

• Checks whether a measure gives similar results when repeated over time, assuming whats being measured hasnt changed

• Tested by giving the same test to the same people twice and comparing results

• Hard to quantify as many factors can change between times of test

Internal Reliability

• Checks whether multiple questions that measure the same concept are consistent with each other

• Tested using statistics like Cronbach’s alpha (values over 0.8 are considered good)

• Ensures your scale or survey items all measure the same underlying concept

Inter-observer consistency

• Checks whether different observers or researchers record or classify things the same way 

• Important for observational studies to ensure results arent biased on whos observing 

Measurement validity

Tool, question or scale actually measures what it is supposed to measure

Face validity

Established at face glance if measure appears to be valid

Concurrent validity

Established if measurement correlates with same criterion that’s relevant to concept 

Construct validity

Established if the concepts relate to each other in a consistent way with researcher’s theory

Convergent validity

Established if a measure of a concept correlates with a second measure of the concept that uses a different measurement technique

Relationship between reliability and validity

• Not reliable, not valid => results are inconsistent and don’t measure what they should

• Reliable but not valid => consistent results, but they’re measuring the wrong thing

• Reliable and valid => consistent and accurate

• A measure that is not reliable will not be valid

• A measure that may be reliable but not valid

Advantages of open questions

• Allows for replies the surveyor would not have thought of

• Makes it possible to tap into participants knowledge

• Particular important issues for respondents can be examined

• Can generate fixed-choice format answers

Disadvantages of open questions

• More time consuming 

• Answers must be coded

• Less convenient to create an answer

• Intra-interview variability

Advantages of close questions

• Decreases intra-interview variability

• May be easier to understand question because answers are provided 

• Can be answered quickly and easily which reduces response time/rate

Disadvantages of close questions

• Loss of spontaneity and authenticity because important answers may not be an option

• Respondents may not understand wording of question

• Respondents may not find an answer that applies to them

How to conduct a structured interview

1. Know interview schedule

2. Prepare the introduction (provide rationale)

3. Create a bond with the interviewee but have a balance

4. Introductory statement 

5. Probing (ask for more detail when needed)

6. Prompting (give hint or examples to get answer from interviewee)

7. Assess interviewers 

Interviewer and Researcher errors

• Interviewer error => misreading, skipping or rewording a question that can change its meaning

• Recording mistakes  => errors in writing or typing responses

• Data entry errors => mistakes when transferring data into a computer file

• Bias from interview characteristics => respondents may answer differently depending on the interviewers traits

• Intra-interviewer variability => inconsistency within or between interviewers which affects reliability 

Bad questions

• Poorly worded or ambiguous

• Double-barred => asking two things at once

• Leading or biased wording => suggesting a certain answer

• Negatively phrased or double negatives => hard to understand 

• Unbalanced response options

Respondent issues

• Misunderstanding of the question => interprets the question differently

• Memory lapses => people may not accurately recall past events 

• Response sets => answering in a pattern

• Social desirability bias => giving answers that makes them look good rather than truthful

Questionnaire design and format issues

• Complex layout => unclear instructions or poor formatting reduce completion rates

• Questions order effects => earlier questions can influence answers to later ones

• Lack of clarity => since there’s no interviewer to explain, can confuse respondents

• Missing data => due to skipped questions

• Inappropriate length => overly long surveys discourage participation 

Mode of survey administration issues 

• Telephone => may exclude people without phones, hard to sustain long interviews, may get new number

• Online surveys => may exclude older adults or those without internet

• Face-to-face interviews => may introduce bias or social pressure

Sampling and representation problems

• Sampling error => sample might not perfectly represent whole population

• Non-response bias => certain groups may be less likely to respond

• Coverage error => missing parts of the population due to survey method

Problems with secondary data

• Lack of familiarity with someone else’s dataset

• Complex data structure => hard to interpret

• Missing key variables => needed for your study

• Ecological fallacy => using group-level data to make claims about individuals

Probability 

Uses random selection methods, quantitative 

Non-probability 

Does not use random selection methods, qualitative

Sources of bias in sampling

1. Not using random method to pick the sample

2. Sampling frame

3. Non-response

Sampling error

difference between the results from your sample and the true results of the whole population

Four types of probability samples

• Simple random sample

• Systematic sample

• Stratified random sample

• Multi-stage cluster sampling

Simple random sample

• Each element has the same probability of being selected

• Element = individual

• Number all the elements starting from 1

• Sampling ratio = n/N (percent of population they sampled)

Systematic sample 

• Selected directly from sampling frame

• Samples in intervals 

• Ex.30,60,90

• Periodicity => sampling frame are arranged in some systematic order

Multi-stage cluster sampling

• For large population (no adequate sampling frame)

• Involves two or more stages 

  • Selecting clusters

    • Then selecting subunits within cluster

• Technical complications (not all clusters are the same size)

Issues with sampling size

• Non response

• Heterogeneity of population

  • Greater the heterogeneity of the population on characteristics of interest, the larger the sample size should be 

• Kind of analysis 

  • Sample size may vary depending on what sort of analysis will be done

Types of non-probability sampling

• Convenience sampling

• Snowball sampling

• Quota sampling

Convenience sampling

• Cases are included because they are readily available 

• Selecting people who are easy to reach

• Problem: One cannot generalize the results to some larger population with any confidence

• Useful for pilot studies and testing reliability 

Snowball sampling

• A form of convenience sampling 

• Researcher makes contact with some individuals, who in turn provide contacts for other participants 

• Good for rare diseases

Quota sampling

• Researchers set quotas (targets) for certain groups to make the sample look more like the population but still chooses participants non-randomly

• Strengths =>

  • Cheaper and quick

  • Good for pilot tests

• Weakness=>

  • Not likely representative

  • Judgement about eliginility may be incorrect

  • It is not appropriate to calculate a standard error term from a quota sample

Structured observation and sampling

• Often no sampling frame

• May involve time sampling

• May include place studying 

• May include behaviour sampling 

limits to generalization

Generalized only to the population from which the sample was taken 

Reducing non-response

• Telephone interviews

  • Call back

  • Reassure participants that you are not out for material gain

• Face to face contact

  • Dress appropriately 

  • Be flexible to accommodate participants

• Mailed questionnaires

  • Good cover letter

  • Make personal

  • Stamped return

Data analysis 

• Data preparation

• Coding

• Checking

• Data cleaning

• Preparation of variables/categories

Descriptive statistics 

• Part of statistics concerned with the description and summarization of data

• Used to organize and describe a sample

• Describes basic patterns in the data

Inferential statistics

• Concerned with the drawing of conclusions from data

• Used to extrapolate from a sample to larger population

• Allows researcher to make precise statements about the level of confidence they can have

Univariate analysis

• One variable at a time

• First step is to create frequency tables for the variables of interest 

• Measures of central tendency 

• Measures of dispersion:

  • Range, percentiles, standard deviation

Measures of central tendency

• Way of summarizing the data using a single value that is in some way representative of the entire data set

• Tells us about the typical value in distribution

Mean

• Average

• Most applicable to nominal data

• Distorted by unusual values called “outliers”

Median

• Score that divides the distribution into equal halves

• “Middle point”

• Not affected by outliers

Mode

• Number or event that occurs most frequency in a distribution

• One data set can have many modes

• Unaffected by outliers 

Positively or right skewed

Occurs when there is cluster of lower values, the smaller, more spread out tail will be on the right

Negatively or left skewed

Occurs when there is a cluster of higher values, the smaller more spread out tail will be on the left

Bivariate analysis

• Determines whether there is a relationship between two variables

Contingency tables

• Allows simultaneous analysis of two variables

• Identify patterns of association

• Can be used for any variable type

• Normally used for nominal or ordinal data

Pearson’s r

• Normally used with internal/ratio data

• Values from 0 (indicates no relationship)

• to +1 (indicates perfect positive relationship)

• or -1 (indicates perfect negative relationship)

Kendall’s tau-b

Shows correlation between pairs of ordinal variables, or with one ordinal and one interval/ration variable

Spearman’s rho

• Shows correlation between pairs of ordinal variables

• Like Pearson’s r, values range from 0 to +1

Cramer’s V

• Shows the strength of the relationship between two nominal variables

• Values range from 0 to 1

• Usually reported with contingency table and chi-square test

Test for statistical significance

• Make null hypothesis

• make an acceptable p-value

• If the null is correct there is no relation

Two types of errors for statistical significance

• Type I: rejecting a true null hypothesis

• Type II: not rejecting a false null hypothesis

Chi-square Test

• Used with contingency tables

• Measures the likelihood that a relationship between the two variables exists in the population

• Calculated by comparing the observed frequency in each cell with what would be expected by chance

Multivariate analysis 

• Examines the relationship between three or more variables

• Can be used to test for spuriousness (exists if two variables are correlated but only through a third variable)

• Can be tested for intervening variables ( X => Y => Z)

• Can be used to test interactions (if the effect of one independent variable varies at different levels to that of a second independent variable 

• Used in multiple linear regression (how much of the variation in the dependent variable is explained by the independent variables

• To illustrate must consider bivariate regression (involves one independent variable and one dependent variable)

  • Now add a second independent variable and move to multivariate regression