Midterm

What are the three main categories of research?

1. experimental

2. exploratory

3. descriptive

experimental type of research def

How well are the variables controlled?

exploratory type of research def

How do these variables relate to each other?

descriptive research type def

What does this observed phenomenon mean?

What is the highest and lowest levels of hierarchy of evidence?

highest: level 1a - SR/MA of RCT's

lowest: level 5 - clinical experience, expert opinion, mechanism-based reasoning

Why is level 1a high on the hierarchy of evidence?

because there is a lot of controls

Should a PT continue a treatment that aligns with a patient's values and our expertise and seems effective, even if there is no evidence to support it?

Yes, because the treatment hasn't been shown to cause harm

What are the three principles of ethics?

respect for persons

beneficence

justice

respect for persons (principles of ethics) def

informed consent

autonomy, protection of vulnerable populations

Who are vulnerable populations with respect to persons?

children, older people, inmates, pregnant women

beneficence (principle of ethics) def

that the benefits for the patient outweigh the risks

justice (principles of ethics) def

fairness in the distribution of benefits/burdens

outcome of the nuremberg code in 1949

first formal guidelines

voluntary consent to participate

outcome of the declaration of helsinki in 1964

independent review of protocols

outcome of the national research act of 1974

institutional review board (IRB)

outcome of the belmont report in 1979

the common rule

What does the common rule include?

respect for persons

beneficence

justice

What are the 4 levels of measurement?

nominal, ordinal, interval, ratio

probability def in sampling methods

An element of random chance, someone can be selected

What are the 2 outcomes in probability?

improve the generalizability of the results

less feasible (harder to recruit but better for results)

Nonprobability in sampling methods

no element of random chance

What are the 2 outcomes in nonprobability?

limited generalizability (population can be over-represented)

more feasible

Example of simple random sampling

flipping a coin or spinning a wheel

Example of systematic sampling

started randomly, but they have a system

ex: start randomly, but they pick every 6th person

Example of stratified sampling

people are divided by trait rather than randomly picked

Example of convenience sampling

picking people that are convient to you

When do you use stratified or quota sampling?

When you want the representation of a subgroup

When do you use cluster sampling?

when the population is geographically dispersed

When do you use purposive sampling?

when you need specific characteristics

When do you use a convenience sample?

When you want to recruit everyone eligible as they arrive

what is operational definitions?

how you turn an idea/concept into something you can measure

multi-factor vs multi-variant

multi-factor has 2 or more INdependent variables

multi-variant has 2 or more DEpendent variables

simple def of reliability

getting similar results when nothing has changed

intra-rater reliability def

same person measuring the same thing at different points in time

inter-rater reliability def

different people measuring the same thing at the same time

What is relative reliability

consistency of individual scores relative to each other

What are examples of relative reliability?

intra-rater, inter-rater, test-retest, internal consistency

What do we use to measure relative reliability?

measure with ICC/Kappa, and Cronbach's alpha

Does relative reliability quantify how much a score will vary for an individual?

NO

What is absolute reliability?

the consistency of an individual score, regardless of how it compares to other scores

What are examples of absolute reliability?

standard error of measurement

ties to MDC

Do reliability coefficients have units?

No - they are useless

In the equation rxx = true score variability/true score variability + error variability, what does the top mean?

The difference between individuals

In the equation rxx = true score variability /(true score variability + error variability), what does the bottom term mean?

difference within individuals

When error makes up a large part of the true score variability, what happens to the reliability?

It deflates the reliability

Does rxx become larger when the true variance increases?

Yes, as long as the error is held constant

Can an outcome measure be very reliable but not responsive?

yes but it isn't very common

MDC def

value at which the observed change is NOT measurement error

What is minimal detectable change (MDC)?

When the change is real but not clinically meaningful

What is the minimal clinically important difference (MCID)

when the change is real and clinically significant

MCID def

The ability of an instrument to detect minimally important change

What is the relationship between internal consistency and split-half reliability?

Slip-half reliability is a measure of internal consistency

simple definition of validity

when the measurement is right (or compared to the gold standard)

difference between reliability and validity?

Reliability = consistency of a measurement

validity = alignment of the measurement with a targeted construct

What is face validity?

When an instrument appears to test what it is intended to test,

It is judged by the users of a test after the test is developed

Face validity is judged by those who _____ the instrument

USE

What are the 3 C's of validity?

contect

construct

criterion-related

What does construct validity look at?

If measurements represent the concept

What is content validity used for

usually in questionnaire development

Content validity is judged by those who ____ the instrument

DEVELOP

What is the similarity between face validity and content validity?

There are no statistics to measure either type of validity

Criterion-related validity def

Can the outcomes of the instrument or index test be substituted for an established gold standard

What are the 2 types of criterion-related validity?

concurrent and predictive validity

concurrent validity def

Scores from a new measure correlate with scores from a well-established measure administered AT THE SAME TIME

predictive validity def

establishes that the outcome of the target test can be used to predict a future criterion score or outcome

Construct validity def

reflects the ability of an instrument to measure the theoretical dimensions of a construct

What are the 3 types of construct validity?

known-group method

convergence

divergence

What does the known-group method test?

tests if the results differ between 2 groups that are known to be different

What does convergence show

If there is a correlation between 2 constructs

What does divergence show?

show that there isn't a correlation between 2 constructs

What mistakes are made when researchers interpret validity evidence

1. treating it as "all or none"

2. assuming reliability also implies validity

3. confusing a strong correlation as "proof"

4. Overgeneralizing weak evidence

Can reliability and validity change over time

yes

How does reliability change over time

Raters become trained, equipment wears out, and testing procedures change

How does validity change over time

new evidence can emerge that shows the measurement behaves differently than expect, or clinical practice evolves

What are the characteristics of a true experimental design?

1. The independent variable is manipulated by the experimenter

2. Include a control or comparison group

3. Participants are randomly assigned to groups

What is external validity?

What can you do with the results outside the study

What is internal validity?

What happens inside the study to affect the causal relationship

What are threats to internal validity?

testing

instrumentation

maturation

history

selection

regression to the mean

What is random sampling

How do we obtain the people participating in the study

What is random assignment?

How we form our groups in our study

What is concealed allocation?

not knowing which group someone is assigned to when eligibility is determined

What is random assignment (in relation to concealed allocation)

not knowing which group someone will be in when determining intervention