scientific inquiry

research question, research subjects, and tools available to us

what does the type of data we gather depend on

indirect and incomplete

measurements are often what

directly

things that we care about often can’t be measured as what

height, weight, color, brightness, distance, force, acceleration, time, angles, electrical potentials, etc

what are examples of direct measurements

people’s abilities and feelings like mental function, strength, pain, or health

when do measurements become less direct due to the abstract nature of the things we care about

constructs

abstract variables that are a combination of variables that we can measure

operationalize variables

because of the indirect nature of measurement, what must be done

defining constricts into measurable terms

what does operationalizing variables mean in regards to indirect measurements

clarity, precision, and replicability in research

what does operationalizing variables crucial for ensuring

different scales of measurement

what is used for different types of variables

the tools and measurements taken

what does the precision of measurement depend on

discrete variables

measured in whole units (ex. number of children)

dichotomous

only two possible categories exist (ex. gender, smoking status, yes/no questions)

polytomous

more than two possible categories exist

continuous variables

can by any value along a continuum ( ex. height and weight)

hyper-precision or inclusion of additional numbers

when using continuous variables, what should you be careful to avoid because it wouldn’t add any additional information

instrument

what is the precision of a continuous variable based on so we limit them to

nominal, ordinal, interval, and ratio

what are the scales of measurement

nominal/ categorical

scale of measurement that assigns names or categories to items; if you change the name the thing itself still doesn’t change

ordinal

scale of measurement that assigns names in a manner that indicates rank order (ex. the place one finishes in a race); defines the rank-ordered position within some distribution but does not describe the difference between each position

ordinal

what scale of measurement is considered poor form by purists to perform mathematical operations on so instead, frequency counts are taken for each name or numeral and compared between groups

interval

scale of measurement that assigns numeric values of equal intervals but does not have a true zero and so ratiometric comparisons are not possible (ex. temperature measurements: 10 and 15C has the same temp difference as 20 and 25C and OC does not mean that there is no temperature at all)

ratio

scale of measurement that assigns numeric values of equal intervals and there is a true zero (ex. temp measurements in kelvin

nominal

if there is not a clear quantitative relationship between different values then what scale of measurement is it

ordinal

if there is not a consistent interval between the different values, then what scale of measurement is it

interval

if there is not a true zero for the different values, then what scale of measurement is it

frequency, percentile ranks (median), means and differences, ratios

what are the different types of summary statistics that apply to different types of data

frequency

is the number of times a value appears in a data set and is applicable to all types of data but is only useful in nominal and sometimes ordinal data

median and percentile

used for all quantitative data (everything except nominal)

means and differences (add/sub, mean, error)

analysis that can only be used for interval or ratio data

ratios

can only be calculated with ratio data

ordinal

manual muscle testing is an example of what type of measure of muscle strength

ratio

maximal voluntary isometric contraction using a strain gauge is an example of what type of measure of muscle strength

correlation

analyses the degree to which two variables change in a related manner but does not tell us about how similar the values are because it shows covariation and not matching

causation

what does correlation not tell us about between variables

extraneous variable

factor in a study that is not the main focus but still affects the dependent variable or study outcome; unknown factors driving observed correlations

spurious correlation

when two variables may correlate by chance and happens about 5% of the time, assuming the typical type 1 error rate

actual values

variables that correlate can have different what because covariation does not equal similarity

correlation coefficient (r)

the strength of covariation between two variables and range from -1 to +1 with the absolute value indicating the strength of the relationship while the sign indicates the direction, 0 being no relationship

pearson product-moment

type of correlation coefficient used for linear relationships and interval/ratio data through two continuous variables and is abbreviated with r

spearman

type of correlation coefficient used for two continuous variables that are ordinal or monotonic curvilinear (p or Rs)

point biserial (Rpb)

type of correlation coefficient used for one continuous variable and one dichotomous variable

phi coefficient

type of correlation coefficient used for two dichotomous variables

0-0.25

what is the ballpark figure for r that indicates a strength of little to none

0.25-0.50

what is the ballpark figure for r that indicates a strength of fair to moderate

0.50-0.75

what is the ballpark figure for r that indicates a strength of moderate to good

0.75-1

what is the ballpark figure for r that indicates a strength of good to excellent

multivariate traits that contribute to constructs

what is unlikely to correlate well with any single variable (ex. personality), so we can accept lower correlation coefficients

fewer variables

traits that are determined by what should correlate better with each variable because we expect to have a simple, direct relationship and so require higher correlation coefficients (ex. resting membrane potential)

sample size and number of contributing factors

the r value depends on what

decrease sampling error

what does increasing sample size do

lower r values

what does better approximating the underlying population allow us to accept

degrees of freedom

is the number of independent pieces of information in a data set that are free to vary when estimating a statistical parameter or evaluating a system; allows

degrees of freedom

calculated as n-2 and you can draw a straight line between any two points

three

how many values do you need to have a chance to deviate from the fit line

the appropriate critical value when performing statistical tests like the pearson correlation

what does degrees of freedom help determine

error or flaws

what do all measurements contain and refers to the difference between measurement and the true value of whatever we are talking about

validity

the ability of a test to measure what is intended to measure

reliability

the ability to have the same results every time we take the measurement so it is consistent between different trials

reliability

an increase in error decreases what

truth and error

a measurement includes both what

point estimate

a single numerical value calculated from sample data that serves as a best guess for an unknown population parameter; as close as you can get to the truth

it is valid and reliable

what does it mean if there is no error in a measurement

instrument

potential source of error due to the tool(s) used to take the measurement which may be caused by environmental conditions that affect it, improper calibration, or inherent noise in measurement taken

rater

potential source of error from the person taking the measurement because they may read the value incorrectly or use instrument inconsistently

subject

potential source of error due to the person or the characteristic being measured; participates may provide incorrect information or are influenced by knowledge of the experiment like group allocation

random errors

occur by chance in both directions and is unpredictable and inconsistent; reduces reliability but does not affect validity

random errors

measurements are variable but the average is accurate

following strict protocols for taking measurements

since random errors can not be avoided completely, what helps minimize them

systematic errors or bias

occur in one direction and either consistently over or under-estimates the true value; reduces the validity of measurement but does not affect reliability

systematic errors or bias

measurements are consistent but average is inaccurate, indicating an issue with the measurement

properly calibrating instruments, following strict protocols for taking measurements, and blinding researchers and subjects

how can systematic bias be reduced

regression toward the mean

by taking multiple measurements, random error or extreme values get canceled out and the average of those measurements approaches the true value is described as what (very high scores tend to decrease and very low scored tend to increase

baseline equivalence

regression toward the mean highlights the importance of what in order to compare groups

whether the measurement requires judgement and whether teh same tool/rater makes both measurements

the type of reliability is determined by what

rater

when scoring requires interpretation or judgement, what do you asses the reliability of

intra-rater reliability

compares measurements by the same rater

inter-rater reliability

compares measurements from different raters

same trials if possible

ideally, in inter-rater reliability, should raters observe the same or separate trials

instrument

when scoring does not require interpretation, what do you assess the reliability of

test-retest reliability

compares measurements on the same instrument

alternate forms reliability

compares measurements on different versions of the instrument; different questions but same character (SAT) and is usually measured with correlation coefficients

test-retest reliability

varies depending on the stability of the variable and should be long enough to avoid potential effects of repeated testing like fatigue or learning

testing effect

the ability of an initial test to improve subject performance on the following test; aka practice improves performance

testing effect

any test that involves skilled movements or cognitive tasks are sensitive to what

nominal/ordinal

what is kappa generally used for

interval/ratio

what is ICC generally used for

coefficients kappa or ICC

what is reliability usually represented as

detectable change

what is easier to interpret in regards to reliability

measurement error

repeated measures allow us to estimate what

reliability coefficient

is the ratio of error that is due to the inherent error of our true value (Et) and has a theoretic range of -1 to 1 but negative values are never reported so there is a practical range of 0 to 1; based on correlation

better reliability

larger reliability coefficients indicate what

reliability

measurement error worsens what

Et

measurement variability (ex. people having different heights)

Em

measurement error (ex. recording wrong height)

intra-class correlation (ICC)

type of reliability coefficient used for continuous variables and interval ratios to estimate reliability and requires agreement and

kappa statistic

type of reliability coefficient used for dichotomous variables and estimates agreement beyond the level expected by chance

correlation assumes a linear relationshup between two sets of numbers (y=mx+b) where y and x can be vastly different while ICC assumes a linear relationship between two sets of numbers (y=x) so we expect the numbers to agree with each other in order to be reliable

what is the difference between ICC and correlation coefficients

minimal detectable change

the smallest amount of difference in a value that passes the measurement error; calculated based on the ICC and 95% confidence

becomes smaller so more willing to trust smaller changes

when measurements are reliable, what happens to the MDC