Chapter 8: Quantitative Methods

AC Copy

the population

What does N stand for?

the sample

What does n stand for?

Yes, depending on the focus of the study.

ex. population = MU students and sample = OT student at MU

• focus of study changes

• now the population = college students in NEPA and sample = MU students

Can the sample become the population and vice versa?

OT juniors at MU = sample

OT students at MU = population

Given these two variables, which is the sample and which is the population?

• OT juniors at MU

• OT students at MU

• describe the study population and sample characteristics

• describe the study procedures and variables

• describe the instruments that were used

• describe the data analytic plan (nominal/ordinal, interval/ratio, non-parametric/parametric)

How do you put quantitative research in context?

when you have a nominal/ordinal scale

What are non-parametric methods used for?

when you have an interval/ratio scale

What are parametric methods used for?

the intended audience (other scientists, OTs, PTs, SLPs, doctors, etc.)

What is part of the formal structure of quantitative research?

Survey design

a quantitative design that describes trends, attitudes, or opinions of a population

survey design

a quantitative design that tests for association and studies a sample of the chosen population

• descriptive questions

• questions about relationships between variables

• questions about predictive relationships between variables over time (longitudinal studies)

What 3 types of questions should survey designs answer?

experimental design

quantitative design involving systematic manipulation of one or more variables to evalutate an outcome

experimental design

quantitative design that holds other variables constant to isolate effects (experimental and control group have the same conditions except for that one you are changing. ex. the treatment)

experimental design

quantitative design that generalizes the results to a broader population

• the survey design

• the population and sample

• instrumentation

• variables in the study

• data analysis and interpretation

What are the components of a Survey Study Method Plan?

validate the survey (when developing your own survey)

asking people who have more expertise in developing surveys about the quality of your own survey

• provide a purpose for why you are using survey research

• indicate why the survey method is preferred

• indicate type of survey design (cross-sectional or longitudinal)

• specify form of data collection and rationale (ex. phone, mail, internet, interview)

What do you need to state in the survey design? (within a survey study method plan)

• cross-sectional (data collected at one point in time)

• longitudinal (data collected over time)

What are the two types of survey design?

telephone, mail, internet, personal/group interviews

What are the possible different forms of data collection in a survey design?

single-stage

Which sampling design? researchers select the final sample directly from the entire population in one step

multi-stage

Which sampling design? researchers select samples in multiple steps or stages

systematic

What type of sampling? researchers select participants using a fixed interval (pattern) ex. every 8th person is selected

random

What type of sampling? every individual in the population has an equal chance of being selected

stratification

ensuring specific population characteristics (ex. gender) are represented

• done by dividing the population into groups and samples are taken from those groups

power analysis

• use this in population and sample component of survey study method plan

a statistical method used to determine the sample size needed to detect significant effects or associations

• construct validity

• content

• predictive

• concurrent

4 types of validity described in the instrumentation component

construct validity

type of validity: the extent to which a test actually measures the concept or construct it is supposed to measure

• measures the right concept (ex. a spoon that measures exactly 5mL)

content validity

type of validity: the extent to which a test covers all parts of the topic or concept it is meant to measure

• covers the full topic (ex. measures volume)

predictive validity

type of validity: how well a test predicts future outcomes or performance

• predicts the future

concurrent validity

type of validity: how well a test correlates with another established measure taken at the same time

• agrees with another test now

internal consistency

measures whether all the questions in a test measure the same concept. (ex. all questions relate to anxiety, the focus of the study)

test-retest reliability

measures whether a test gives similar results when taken at different times (ex. a person takes a personality test today and two weeks later)

• cover letter

• items (demographics, attitude items, behavior items, factual items)

• closing instruction

• type of scale for responses

What are the parts of the major content to include in the instrumentation section?

pilot testing/field testing (in instrumentation)

a small trial run of a survey or study used to identify problems and improve questions or procedures before the full study

computer programs

What are used for data analysis?

1) Number who did and did not respond

2) Method to determine response bias

3) Plan to provide descriptive analyses (the participants and descriptions (characteristics) of them)

4) Calculate total scale scores

5) Statistics and program for inferential statistical analyses (allows you to infer based on the data you collect, gives you predictive information, direction)

6) Present results in figures or tables and interpret

What are the steps used for data analysis?

• Report how the results answered the research questions (qualitative) or the hypothesis (quantitative)

• Practical evidence in terms of effect size and confidence interval

• Discuss implications: consistent with, refute, extent previous studies

What do you include in the interpretation of results and the discussion section?

• Participants

• Variables

• Instrumentation and materials

• Experimental procedures

• Threats to validity

• Data analysis

• Interpreting results and writing a discussion

What things do you include in an experimental method plan?

inclusion criteria group

Which is decided first? inclusion or exclusion criteria group?

inclusion: stroke patients at GVH

exclusion: stroke patients under the age of 65 at GVH

Which is the inclusion criteria and exclusion criteria of this scenario?

• stroke patients under the age of 65 at GVH

• stroke patients at GVH

inclusion criteria

characteristics that participants must have to be included in the study

exclusion criteria

characteristics that disqualify someone from participating

random/randomization (selection of participants)

everyone in the sample or population has an equal chance/opportunity to be chosen to participate

nonrandom selection of participants

convenience, whoever wants to participate, you will take

True experiment

individuals randomly assigned to groups

• involves Randomized Control Trial (RCT)

Quasi-experiment

partial or no control over random assignment

• does not involve RCT

a formal experimental design statement

What should you end an experimental study method plan with?

• the independent variables

• include a manipulation check measure

• the dependent variable

• other variables measured (confounding, those that contribute to noise)

What do you need to identify with variables?

pre-experimental

type of experiment: no random assignment; weak control; often just one group

true experiment

type of experiment: has random assignment and control groups; strong evidence for cause-and-effect

quasi-experiment

type of experiment: no random assignment; uses existing groups; weaker control

single-subject design

type of experiment: focuses on one participant (or a small number) over time; often used in clinical or behavioral research

a visual model

ex.

• X = treatment

• O = observation

• R = random assignment

What should you provide to illustrate the research design used?

within-group comparison

type of comparison: same group tested before and after treatment

between-subjects comparison

type of comparison: different groups receive different treatments and are compared to each other

external validity (threat to validity)

drawing incorrect inferences from sample data to other persons, settings, situations

internal validity (threat to validity)

procedures, treatments, or experiences of the participants that threaten inferences in experiments

P-value

the probability of obtaining results at least as extreme as the ones observed, assuming the null hypothesis is true

• whatever occurred happened purely by chance

• statistically significant

• we can reject the null hypothesis

What can we say if the p-value is 0.05 or less?

• not statistically significant

• cannot reject the null hypothesis

What can we say if the p-value is more than 0.05?

You reject a true null hypothesis and get a false positive conclusion.

What is a Type I error?

alpha (level of significance)

the probability of making a Type I error

if you get a p-value above 0.05 and cannot reject the null hypothesis

When does a Type I error occur?

accepting a false null hypothesis

What is a Type II error?

beta (depends on n (sample size) and o (variance))

the probability of making a Type II error

1 - beta (power of the test)

What is the probability of rejecting a false null hypothesis?

increasing n

How can you increase power?

to reject the false null hypothesis

What is the researchers’ goal?

internal threats to validity

procedures, treatments, or experiences of the participants that threaten inferences in experiments

external threats to validity

drawing incorrect inferences from sample data to other persons, settings, situations

statistical conclusion of validity

inadequate statistical power or violation of statistical assumptions

construct validity

inadequate definitions and measures of variables

t-test

test that compares the means of two groups

ANOVA test

test that compares the means of three or more groups

descriptive statistics

What are these examples of?

• mean, SD, ranges

mode

value that occurs most frequently in a data set

Pearson Correlation Coefficient

measures the linear relationships between variables