OT 575 - research methods part 10

how do questionnaires differ from interviews?

questionnaires use a paper-and-pencil or digital format; interviews use oral methods

what is the first step in constructing a questionnaire/interview?

defining your objectives

what are the four types of questions that can be asked in a questionnaire/interview?

open-ended questions, closed questions, rank-order questions, and categorical responses

what is the delphi method?

uses a series of questionnaires to achieve consensus about an issue

what are the four major categories of questions?

hypothetical, devil's advocate, ideal position, and interpretive questions

what is an example of a hypothetical question?

"let's support this is my first time collecting data. what would it be like?"

what is an example of a devil's advocate question?

"some people say that hybrid teaching is better than in-person teaching. what would you say to them?"

what is an example of an ideal position question?

"what do you think the ideal study design would be for a children's study?"

what is an example of an interpretive question?

"would you say that the online teaching experience is different from an in-person teaching experience?"

what are normative surveys?

surveys that establish norms for abilities, performances, beliefs, and attitudes; must be administered in a standardized manner

what are some drawbacks of using normative surveys?

creates blanket generalizations; analysis and standardization of procedures also create bias

how is sample size obtained?

identify an effect size by using info from similar studies or your own preliminary study

what types of information should be used to determine effect size?

means and standard deviations, correlation, sample size, chi-square values, etc. of dependent variables

how should sample size be determined?

use the most important query among your research questions or hypotheses (also known as "primary hypothesis"); use second most important research question if you cannot find relevant studies

how is sample size determined in a descriptive study (i.e., you are using descriptive stats to analyze your data)?

you do not need to calculate sample size (unless the data is used to compare groups/correlate data)

what types of stats do you need from a previous study or pilot study to use gpower software?

means, standard deviation, mean difference, r correlation (based on your study)

what should the power be set to in gpower software?

.70 or .60 if study is a pilot study (otherwise conventional = .80)

what should the alpha level be set to in gpower software?

.05 (with statistical power of .80)

what statistical test should be used to compare two independent group means?

independent t-test

how is effect size calculated to compare two independent group means?

using a d-index

which stats do you need to know in order to use an independent t-test in gpower software?

means, standard deviations, sample sizes (OR a t-value and sample sizes in a previous study)

what statistical test should be used to compare two correlated group means?

paired t-test

how is effect size calculated to compare two correlated group means?

using a d-index

which stats do you need to know in order to use a paired t-test in gpower software?

means and standard deviations of different scores OR means and standard deviations at each measurement time (pretest and posttest) and r correlation

what formula should be used to determine effect size (if r correlation is not available)?

d' = d/sd

d = mean of difference scores

sd = standard deviation of difference scores

what statistical test should be used to compare three or more independent group means?

one-way ANOVA

how is effect size calculated to compare three or more independent group means?

using an f-index

what constitutes a small effect size for three or more independent groups?

f = .10

what constitutes a medium effect size for three or more independent groups?

f = .25

what constitutes a large effect size for three or more independent groups?

f = .40

which stats do you need to know in order to use one-way ANOVA in gpower software?

between-group variance and within-group variance OR sum of squares for error

how is effect size calculated for correlation?

you do not calculate effect size; effect size (r) is the same as the pearson correlation (r)

what constitutes a small effect size for correlation?

r = .10

what constitutes a medium effect size for correlation?

r = .30

what constitutes a large effect size for correlation?

r = .50

what statistical test should be used to analyze frequencies for two variables?

chi-square

how is effect size calculated in a chi-square test?

ω = √(Χ^2/ N)

(divide the chi-square value by # of scores and take the square root)

what constitutes a small effect size for chi-square?

ω = .10

what constitutes a medium effect size for chi-square?

ω = .30

what constitutes a large effect size for chi-square?

ω = .50

what should be done if outcome measures are missing?

add attrition rate (%) OR expected missing cases then calculate # of participants you need to recruit

attrition rate example

sample size = 30

attrition = .2

need to recruit 37-38 participants (30/0.8 = 37.5)

true or false: sample size can be determined using nonparametric stats.

false; effect size can be determined , but parametric stats must be used to determine sample size