Research Principles - Research questions, hypothesis, reliability, validity

A research question should be

clear and answerable.

What is often the motivation for researching with SLPs working in EBP framework?

you have a client and you don’t know how to treat them so you do a study to figure it out

When choosing a research question, what should you look at?

Other research studies.

Check their weaknesses/limitations, area for future research section, address the functional impact of that research, change the setting or stimuli, look at other fields and apply their practice to SLH.

Formulating research questions you should consider including

How your question will give information about the study

information about variables (observation, nature of the study what type of study)

intent of the research (describing group/conditions, relation between variables, difference between groups.)

What is a hypothesis?

Statement of predicted outcome

What is a null Hypothesis (H0)?

Says that we are not going find significance/relationship/difference between x and y.

Why do we have null hypotheses?

To keep bias out of it by saying there won’t be different unless we prove that there is.

What does it mean to reject a null hypothesis?

Shows there is a significant result (relationship/difference)

Most researchers want to reject the null

What does it mean to fail to reject a null hypothesis?

There is no significant result.

What is an alternate hypothesis (aka research hypothesis, H1)

• Says we will find significance/relationship/difference with X and Y.

An alternate hypothesis can be

directional or non-directional

When a difference study is directional…

The hypothesis is stated in a way that you expect one result to be better/worse than another result

When a relational study is directional..

Hypothesis is stated in a way that you expect a significant relationship (negative or positive.)

If a study is non-directional…

Hypothesis states there will be a difference or a relationship but the differences can either be positive or negative.

What is a conditional statement?

It is an “if-then” statement. Often found in studies that relate to theories.

EBP research questions are often in what format?

• PICO Form

  • Patient/pop/problem

  • Intervention

  • Comparison

  • Outcome

Strong Research Questions…

• Operationalize the variables

• include specific information in the question

Operationalizers could be

• Age range

• Certain scores

• Certain ranges of severity of issue

• Communication methods (AAC)

• Type of issue

What is validity?

It asks whether or not the research accurately measures what you’re testing?

A measure cannot be valid if it is not

reliable.

Face Validity

logical validity, if you think about it does it make sense?

What is the cons of face validity?

 Can be biased based on your thought process.

Content Validity

 involves more formal judgement. Would an expert agree that the content is valid? (content is correct w/o personal bias.)

Criterion validity

Compares against an existing standard/test/measurement. Has two subtypes: concurrent and predictive.

Criterion Validity: Concurrent

Is the comparison from around the same time period?

May see a validity coefficient (0.0-1.0)

Only meaningful if the standard test is valid.

In Criterion Validity: Concurrent, what is considered a weak or strong relationship numerically?

Closer to 0 = low relationship, closer to 1 = stronger relationship.

Criterion Validity: Predictive

Is the comparison with something new/in the future? Can the test predict something?

Construct validity is related to

theories and constructs.

When doing construct validity, what kind of questions should you ask about your study?

do the patterns follow along with the theory/model and the constructs?

What was your theory and why did you pick it?

Look at the results and compare it to your construct/theory. Did this similarity/difference prove your thesis?

Every theory has a

construct that they use to develop assessments/treatments.

What is a construct?

a skill, tied to an emotion, ability or proficiency.

When establishing a construct you should look for

Convergence or divergence.

What is convergence?

Happens if a person performs similarly on test questions or if two different tests have similar content.

What is divergence?

Happens if a person performs differently on test questions that measure different constructions.

Ecological validity

Does this test apply to the “real world” of the person being tested?

What is reliability?

Result repeatability. Asks if the measure gives consistent and repeatable results.

When it comes to reliability, you want

high reliability.

Very important if comparing results of one client/patient

Reliability may not have to be as high if you are comparing groups

A measure can be reliable even if it’s not

valid.

Inter-observer reliability

• Does this test give the same results no matter who gives it?

• Correlation coefficient

Correlation coefficient is the

•  Degree of reliability

• How much do the scores agree between examiners?

• Strong correlation = similar results (what we want.)

Test-retest reliability

• Participants are tested twice (same examiner)

• Do the results of the test have similarities or are they very different within a short period using the same test taker, test, and examiner?

• You need strong test-retest reliability because if it’s poor, the test is the problem and the person is not in fact getting worse.

Parallel forms reliability

• Two different forms of the same construct/assessment

• Participants take both tests at different times.

• Does the participant obtain a similar score on the different forms?

• Can get a correlation coefficient to determine reliability between two forms. You want high consistency.

Split-half reliability

• Administer test once to participants

• Split tests into 2 equivalent forms, get score for each section/half

• Can get correlation coefficient for two halves (Measure of internal consistency)