CSAD 148: Test 2 Study Guide

What is evidence-based practice?

1. The process of applying a scientifically-sound intervention technique to a particular client

2. Clinical decision-making that integrates the application of clinical experience and expertise, recognition of the needs and perspectives of the person receiving services, and identification of the BEST supporting scientific research evidence

What are the 3 components of evidence-based practice?

1. External, scientific evidence

2. Clinical expertise/expert opinion

3. Client/caregiver perspectives

What are barriers to evidence-based practice?

1. Evidence hard to find/ NOT enough of it

2. Lack of training in evaluating research critically

3. NOT enough time and/or access

- When authors submit to a journal and the editor determines if the article might be a good fit

- If so, the article is sent out to volunteer reviewers who write up a review of the article

- The reviewers can do the following with the article: accept it, accept it with minor revisions, revise and resubmit it, or reject it- if the article gets through this process it will be published!

What is the peer-review process?

- A method that uses deduction to test a theory, form a hypothesis, and test the hypothesis in a set of observations

- A search for knowledge concerning general truths

- A process of objectively studying a problem via collection of data

- A process of constant questioning

- A self-correction mechanism for knowledge acquisition

- A HIGHLY-controlled version of clinical decision-making

What is the scientific method?

What does PICO(T) stand for?

- Population

- Intervention

- Comparison

- Outcome

- (Time)

What does the "population" in PICO(T) mean?

Who the intervention is effective for

What does "intervention" in PICO(T) mean?

What you are doing for the client

What does "comparison" in PICO(T) mean?

What is being compared- something vs nothing or something vs something else

What does "outcome" in PICO(T) stand for?

What we want to change

What does "time" in PICO(T) mean?

How long something takes

You have a client with a speech sound disorder who is currently receiving 30-minute, individual, once-weekly treatment sessions for 10 weeks. You're wondering if GREATER treatment intensity (30 minutes, individual, twice-weekly for 10 weeks) would lead to different outcomes in the production of target sounds- what is a PICO(T) question based on this information?

- P: Clients with speech sound disorder

- I: Twice-weekly 30-minute treatment

- C: Once-weekly 30-minute treatment

- O: Production of target sounds (measured by...)

-(T): 10 weeks

What is an example of a PICO question using the following information: Middle-aged, male amputees are suffering phantom limb pain. Researchers are attempting to find out if gabapentin, when compared with a placebo, is effective in decreasing the pain symptoms of these men

- P: Middle-aged male amputees

- I: Gabapentin

- C: Placebo

- O: Decreased pain symptoms

What is an example of a PICO question using the following information: For children on the autism spectrum, is a naturalistic developmental behavioral intervention (NDBI) MORE effective than discrete trial training (DTT) in developing language and self-help skills?

- P: Children on the autism spectrum

- I: NDBI

- C: DTT

- O: Improved language development and self-help skills

What are the levels of evidence from bottom to top (7 points)

1. Animal and lab studies

2. Case report or case series

3. Case control studies

4. Cohort studies

5. Randomized controlled trial

6. Meta-analysis + systematic review

7. Clinical practice guidelines

What is a systematic review?

A review that includes ALL studies about a given topic, includes a research strategy, narrows down the articles to a smaller number of articles based on inclusionary and exclusionary criteria, reviews ALL the identified articles, determines the level of evidence for each article, then formulates an opinion based on ALL the evidence focused on the HIGHEST level of evidence

What is a meta-analysis?

A study strategy that, like a systematic review, formulates an opinion based on ALL the evidence that is focused on the HIGHEST levels of evidence, but in ADDITION, re-analyzes the data from ALL participants in the previous studies using effect sizes

What is a primary study?

A study that involves collection of data

What is an IV in experimental research?

The presumed cause

What is the DV in experimental research?

The presumed effect

What are 2 components of experimental research?

1. IV

2. DV

In experimental research, there is a possibility of finding what?

Causation

In observational research, is there a possibility of finding causation?

No

In observational research, what are 2 possible things you can find?

1. Comparison

2. Correlation

What is an observational design?

When researchers OBSERVE the potential effects of a risk factor, diagnostic test, or treatment strategy WITHOUT trying to manipulate it

What are the 4 steps of writing a research question?

1. Start with "do" or "does"

2. Determine the IV

3. Determine the DV

4. Write a hypothesis

What is the main difference between a descriptive question and an inferential question?

A descriptive question involves ONE variable whereas an inferential question examines 2 (or MORE) variables

What are the levels of measurement in order from lowest to HIGHEST?

1. Nominal

2. Ordinal

3. Interval/ratio

What is information on nominal values?

- Is a name value

- May be in sets of 2 or 3

- Examples: yes/no, yes/no/maybe

- We code nominal values with numbers but they are NOT technically numeric

What is information on ordinal values?

- These are ordered values (think of a Likert scale :)!

- They are on a greater than/lesser than scale

- Ex: strongly agree/agree/neutral/disagree/strongly disagree

What is information on interval values?

- They go in order but with equal spacing (proportional intervals)

- These values are typically already in numbers

- Examples: Temperature, time

What is information on ratio values?

- Go in order with equal spacing

- Have a true 0

- CANNOT be negative

- Examples: Age (in years, months); height (meters, feet); weight (lbs, kg)

The variable that is changed/manipulated

What is the independent variable?

The variable that is measured

What is the dependent variable?

- Another variable that MIGHT change the dependent variable but which the researchers are NOT manipulating

- Ex: When examining the relationship between activity level (IV) and weight gain (DV), age MIGHT be a confounding variable

What is a confounding variable?

What constitutes an experimental study? (3 characteristics)

1. IV (variable that is manipulated)

2. DV (depends on the IV)

3. Possibility of finding causation

What is a treatment in an experimental study?

A procedure designed to change something

A "fake" drug, often a sugar pill that is given to a participant to see if improvements are psychological in nature

What is a placebo in an experimental study?

The group that does NOT receive treatment or placebo and is instead left in the same position as BEFORE the experiment

What is a control in an experimental study?

- An educated guess that specifies a value difference

- Can involve either a comparison OR a correlation

- Ex: The treatment group was MORE successful than the control group

What is a directional hypothesis?

- A hypothesis that predicts that there will be NO significant distinctions between 2 groups; NO difference nor correlation is predicted

- Ex: The treatment group does NOT differ from the control group

What is a null hypothesis?

Between subjects vs within subjects

- Between subjects: Treatment vs control; treatment A vs treatment B

- Within subjects: DV is measured both before and after we manipulate the IV; pre vs post

A type of comparative study that compares ACROSS time WITHIN a group and can ALSO compare BETWEEN groups at MULTIPLE time points

Longitudinal study

A type of comparative study that compares ACROSS groups at the same time, allowing MORE room for outside factors to have an influence

cross-sectional study

Researchers identify people with an existing health problem ("cases") and a similar group WITHOUT the problem ("controls") and then compare then on a task

- Ex: LaSalle and Conture (1995) comparing children who stutter to children who do NOT stutter

Case-control study

Researchers follow a cohort and track exposures to a variable of interest (A cohort is any group of people linked in some way, such as people born within a certain time frame as in a birth cohort)

- Ex: The Early Autism Risk Longitudinal Investigation tracking infant siblings of children on the autism spectrum (cohort)

cohort study

(in assignment): When participants are randomly assigned to treatment or control

What is randomization?

- Controlling the measurement order

- Ex: Which test do you do better on: the first exam of the week or the last exam of the week?

What is counterbalancing?

Repeating a study to see if you get the SAME results in a different sample but of the SAME population at a different time

What is replication?

Moral principles that govern a person's behavior or the conducting of an activity

What are ethics in research?

1. Happened in the 1930's

2. The researchers at the Tuskegee Institute decided that they wanted to look at the long-term effects of syphilis

3. They did NOT tell the participants (poor, Black share-croppers) who had syphilis and who did not, nor did they treat the syphilis, even when penicillin was discovered

4. A whistleblower discovered what was going on and the study was halted

What are the main points of the Tuskegee Syphilis experiment?

1. Researchers randomly assigned undergrads at Stanford who agreed to participate in this during spring break into 2 groups: "prisoners" and "guards"

3. The researchers encouraged the "guards" to abuse the "prisoners"- it got way out of hand

4. The researchers conducted this experiment in an attempt to answer the question: "How do people become Nazis?"

What are the main points about the Stanford Prison Experiment?

1. This study asked the question, "Why do normal citizens do monstrous things?"

2. It tested how far someone would go to obey after being pressured by someone in a white coat, even going so gar as to fatally shock someone for answering a question wrong

What are the main points of the Milgram Obedience Experiment?

1. In this study, researchers went to an orphanage and split the children into 2 groups

2. Half the kids, they berated about their speech and the other half they were super positive about their speech

3. Some of the children who were berated actually developed speech problems as a result

What are some main points about the Monster Study?

What are the 3 basic principles of the Belmont Report?

1. Respect for persons

2. Beneficence

3. Justice

1. It expresses the ethical conviction that the autonomy of individuals should be respected

2. People with diminished autonomy are entitled to equal protection

What are the 2 main points of the "Respect for Persons" principle?

An obligation to protect subjects from harm by maximizing possible benefits and minimizing possible harms

What is the principle of beneficence?

Promoting equitable representation in research in terms of fairly distributing the risks and benefits of research

What is the principle of justice about?

What principle of the Belmont Report does informed consent satisfy?

Respect for persons

What principle of the Belmont Report does the assessment of risks and benefits satisfy?

Beneficence

What principle of the Belmont Report does subject selection/vulnerable populations satisfy?

Justice

What are the 3 factors of an RCT?

1. Random assignment

2. Treatment vs control

3. Between subjects vs within subjects

Comparison vs correlation

A comparison involves a _______whereas a correlation involves an _______

A comparison involves a difference whereas a correlation involves an association

- Ex: the treatment group was MORE successful than the control group (comparison)

- Ex: The number of hours of therapy was positively associated with outcome (correlation)

Characteristics potential participants must have to participate in a research study

What is inclusionary criteria?

Characteristics that disqualify potential participants from being in a research study

What is exclusionary criteria?

The number of subjects participating in a study

What is sample size?

What is statistical power?

The probability that the test correctly rejects the null hypothesis (the probability of a true positive result)

Standards you will use to select study participants

What are selection criteria?

Matching every subject in one experimental group with an equivalent subject in the other group (Ex: Finding a matching non-student for each student in a study)

What is subject matching?

Selecting sample based on "whoever you could get"

What is convenience sampling?

When the sample does NOT represent the population. (Ex: If you ONLY test children whose parents are college professors, this might misrepresent "all children")

What is a sampling error?

A general term used to describe a number of different conditions or factors that cue respondents to provide inaccurate or false answers during surveys or interviews (Ex: those who didn't love and didn't hate the professor do NOT fill out the course evaluation; lying on a survey response about a sensitive topic)

What is response bias?

That which is used to measure a variable (Ex: A test is supposed to measure the student's overall knowledge of the course material)

What is instrumentation?

The items used to evoke a reaction from participants or respondents in a study (Ex: Each question on a test is a stimulus)

What is a stimulus?

That which is influenced by personal feelings or beliefs and is therefore innately biased (Ex: how stressed do you feel right now on a scale of 1-10?)

What is the meaning of "subjective"?

That which is based on measurable facts (Ex: HR/BPM)

What is the meaning of "objective"?

A reproducible inaccuracy that are consistently in the same direction

i.e. due to a problem that persists throughout the entire experiment (Ex: wording bias and question order bias)

What is systematic error?

A chance difference between the observed and true values of something (Ex: A student has a cold on test day, which affects their performance on said test)

What is a random error?

What is observed score (Hint: it's an equation!)?

True ability + random error

Ongoing crisis in the social and life sciences in which scholars have found that the results of many scientific studies cannot be replicated

What is replication crisis?

What are 5 types of bias in research?

1. Observer bias

2. Confirmation bias

3. Cultural bias

4. Wording bias/question order bias

5. Acquiescence bias

Bias that occurs when researchers see what they expect to see, often because they are NOT blinded to group membership

What is observer bias?

Bias that occurs when researchers search for, interpret, and remember data in light of their own hypotheses, even when the data does NOT actually match

What is confirmation bias?

Bias in which researchers make assumptions through their own cultural lens about participants from other cultuers

What is cultural bias?

Introduction of any systematic error into the measurement

What is wording bias/question order bias?

The tendency to agree with or be positive toward whatever the researcher presents; trying to give the "right answer"; this is participant bias

What is acquiescence bias?

What are 4 ways in which researchers can apply the Belmont Report to recruiting participants for their study?

1. Be ethical

2. No coercion

3. Make sure that the participants give informed consent

4. Maintain privacy and confidentiality

Subject selection must have what 3 characteristics across groups to whatever extent possible?

1. Be defined

2. Be defensible

3. Be equivalent

What are 3 questions to ask when assessing the reliability of measurement?

1. Would the data look different if you changed the question a little?

2. Would the data look different if you asked the question at a different time?

3. Would the data look different if a different person asked the question?

How consistently you are measuring your variable

What is reliability?

Consistency across DIFFERENT individuals taking a measurement at the SAME time (Ex: watching any sports using judges relies upon human observers maintaining a great degree of consistency between observers)

What is interrater reliability?

Consistency with which the SAME individual takes measurements at two DIFFERENT times (Ex: A professor grading easier or harder as they go through the stack of exams)

What is intrarater reliability?

The consistency of measures when the same test is administered to the same person twice

What is test-retest reliability?

Focuses on whether different forms of the same measurement instrument, when measuring the same subjects, will produce similar results (Ex: Students in section 1 taking test form A should score similarly to students in section 2 tasking test form B)

What is parallel forms reliability?

How accurately someone is measuring the construct they think they are measuring

What is internal validity?

A type of internal validity that asks if how you are examining your variables is logical (Ex: Your professor asks for feedback on her exam. Your feedback is non anonymous. How willing are you to provide negative feedback?)

What is content/face validity?

Type of internal validity that examines if your measure predicts something meaningful in the future (Ex: GRE score is NOT predictive of grad school completion)

What is predictive validity?

Type of internal validity that examines how well your study lives up to its claims (Ex: Measures of attitudes about exercise should include not only positive thoughts about exercising but also how much the person actually exercises)

What is construct validity?

Type of internal validity that examines if your study agrees with similarly-designed studies (Ex: Measure of risk-taking might correlation with the number of extreme activities someone participates in)

What is concurrent validity?

what concept asks

how well does your study generalize? Does the test

mirror the real-world situation or population?

What is external validity?