Untitled Flashcard Set
Week One - Lecture Two
↳ Emphasis on using sample data to predict the truth about a larger population
↳ To do so, we must include measures of uncertainty, as our calculations are still predictions
Point Estimate - Our best single guess or estimate about a population, it is the exact number we calculated using our sample
Margin of Error: The largest likely error of the point estimate, helps to create a confidence interval
Confidence Interval: The interval that we are confident our value falls within, this is the point estimate +/- the margin of error, this quantifies how uncertain we are about the population
↳ SHORT CI - A short CI signifies little uncertainty regarding our population
↳ LONG CI - A long CI signifies a lot of uncertainty regarding our population
How can you produce more certainty?
→ COLLECT MORE DATA
Sampling Variability - If you conducted your study again, would you obtain the same results? Likely not, because each sample is different and will provide unique results.
Meta-Analysis - A complete analysis of ALL results from all replicated tests, usually summarized in a forest (diamond shaped) plot
Basic Principles of Research Design
Random Sampling:
↳ Must sample randomly, in order to achieve unbiased and generalizable results
Experimental vs. Non-experimental Studies
↳ Experimental studies are controlled tests that include a manipulated independent variable and a dependent variable (outcome observed)
↳ Justifys casual conclusions (relationships between variables)
↳ Non-experimental studies have no manipulation or assignment
↳ Studies are not controlled and therefore cannot justify casual conclusions
Measurement
↳ Construct - The abstract idea we are interested in
↳ Operational Definition - The precise procedure used to measure the values associated with a construct (i.e., a definition for academic success could be a GPA higher than 3.5)
↳ Measured Variable
Reliable Vs. Valid Measurement
→ Reliable means repeatable; similar tests produce same results
→ Valid means accurate; measures what it is meant to measure
Levels of Measurement
Nominal - Categorized into distinct groups but with no order or quantitative value
Ordinal - Categorized into distinct groups that are ranked or have a natural order
Interval - Quantitative data with equal distance between values and no true zero point
Ratio - Quantitative data that has a true zero point
Week Two - Lecture One
Variable Types:
Continuous variable - Can take any of the unlimited numbers within a range
↳ Height, Time spent on task
Discrete variable - Can only take distinct or separated values
↳ Number of eggs in a basket, children in a family
Measures of Central Tendency:
Mean - The average of the values in a distribution, very influenced by outliers especially in small sets
Median - The middle point within a distribution, not as sensitive to outliers
Mode - Most frequent value in a distribution, not sensitive to outliers
Measures of Spread:
Standard Deviation - The average, or typical, deviation from the mean and like the mean it is sensitive to outliers
Inter-quartile Range - Numbers within 25th percentile, 50th percentile, etc.
Measures of Individual Scores
Z-Score - How many standard deviations a value is from the mean (Ex: If SD is 5, and mean is 15 then a value of 20 would have a z-score of 1)
Percentile - The % of data below a certain score (usually mean), range of values 25% below mean
Week Two - Lecture Two
Standard Normal Distribution - Always has a mean of 0 and a standard deviation of 1
Distribution of Sample Means - Like the mean of a sample, the mean of this distribution will give us the population mean (u)
Standard Error of the Mean - The standard deviation of the sample mean distribution, so SEM is basically a measure of uncertainty in the mean, not uncertainty in the people themselves
How to differentiate:
SD tells you how different individual heights of students are but SEM tells you how much the average height would change if you kept measuring different groups of students.
Small SEM - The averages from group to group are very similar → your mean is precise
Large SEM - The averages jump around more → your mean is less precise
Confidence Intervals (CI):
Researchers most commonly choose a 95% confidence level (95% confident the value falls within this range), the interval shows the range. Ex: CI of [-1,9]
Week Three - Lecture One
How do you determine CI when σ is unknown?
Degrees of Freedom, df
↳ df = (N - 1)
↳ The number of independent pieces of information you have to work with.
↳ Ex: You have 5 numbers and their mean must be 10.
If you choose the first 4 numbers freely, the 5th one is forced to be a specific value to keep the average at 10.
→ Degrees of Freedom help us, with the help of sample SD (s) to calculate a CI
→ However, because we are using sample information and “s”, the CI varies from sample to sample
CI Interpretations:
↳ From the Dance: Our CI is always randomly chosen from the dance, it may or may not capture the true population mean (u) but it has a 95% probability of doing so
↳ Cat’s Eye Picture: A CI takes a cat’s eye shape, essentially saying that the tail ends are less likely to include the mean, and the wider middle section is most likely to include the mean
↳ MoE gives precision:
• Larger MoE [30, 60] means lower precision
• Smaller MoE [44,48] means higher precision
↳ Prediction for Replication: Provides useful information about replicated studies, on average there's an 83% chance that a 95% CI will capture the next mean
Week Three - Lecture Three
Null Hypothesis - The skeptical hypothesis that states there is no effect or difference
Alternative Hypothesis - The hypothesis that predicts an effect or difference does exist
Null Hypothesis Significance Testing (NHST)
STEPS
Formulate a null hypothesis: “The mean IQ of Guelph students is equal to 100”
Collect a sample of data from population (random group of students from UofG)
Evaluate likelihood of obtaining your data if the null hypothesis were in fact true → Given mean and SD from sample, how likely is it that your sample came from a population with a mean of 100 (ensuring accuracy)
P-values: Likelihood of obtaining results IF the null is true
Tells us how strongly our results provide evidence in favour of the alternative hypothesis, with smaller p-values indicating that the observed effect is less likely to be due to chance and more consistent with a real effect
Tcritical vs. Tobtained
Tcritical - The cutoff value that determines significance, depends on significance levels, df, etc.
Tobtained - The t-value calculated from the data
If Tobtained is LARGER than Tcritical, you reject the null hypothesis (it is significant)
Types of Errors
↳ Type I error: False positive, only possible if you're rejecting the null (significant difference)
↳ Type II error: False negative, only possible if you're failing to reject the null (no significant difference)
Beta (b) - Odds of making a type II error
Determined by:
↳ Sample size, more people lowers risks of biased, inaccurate results
↳ Higher alpha, this will lower beta but increases risk of false positive (type I)
↳ Effect size, larger the effect that is being studied the less likely inaccurate results are
Power:
↳ Our chance of correctly rejecting the null hypothesis, if there is truly no effect
Power is influenced by:
↳ Effect Size – the larger the effect, the greater our power to detect it
↳ N – the larger the sample the more likely we can detect an effect if it exists
↳ alpha - the larger our alpha criterion, the greater our power to detect an effect if it exists
Week Four - Lecture One
Independent Groups Design - Participants are assigned to two conditions
Overlap of CIs on Independent Group Means
Gap - P < .01
Just Touching - P ≈ .01
Moderate Overlap - P ≈ .05
Larger Overlap - P > .05
Cohen’s D: A standardized measure of effect
↳ Cohen’s d is a standardized effect size that shows how far apart two means are in standard deviation units
↳ Raw difference (lbs or cm) cant be compared across variables
To measure Cohen’s d:
d = (Effect size in original units) / (An appropriate standard deviation)
So, for an independent groups design:
d = (M2 - M1) / s
Cohen’s d - Interpretation:
Small - 0.2
Medium - 0.5
Large - 0.8
Week Seven - Lecture One
Paired design - Each participant is tested against two conditions, and the important difference is not between overall means but between the pretest results and posttest results for each participant
↳ The IV for a paired design is a within group variable, or repeated measure
↳ Effect size is mean difference, Posttest - Pretest
↳ Paired designs give a more precise CI than CI for separate means, as paired designs remove a lot of noise or influence from between-person variability (ex: mood, ability, etc.)
Independent vs. Paired design:
↳ With independent groups, we can eyeball/estimate our p-value based on the overlap between CIs but with paired design there is no
↳ This is because independent groups CIs depict differences between people, but paired designs are focused on differences within the same person, so you cannot use CI overlap to eyeball a p-value
Cohen’s d - Paired Designs
↳ Basic cohen’s d : (effect size in original units / appropriate standard deviation)
↳ However, for paired designs we must choose the average standard deviation, rather than just pretest or posttest SD’s
So the cohen’s d equation for paired design is: Mdiff / Saverage
↳ Do not mistake this for Sdiff (S2 - S1) as this would produce a much too large d value, so it doesn’t work)
Interpreting the Paired Design
↳ Carryover Effect - Any influence of the first measurement on the second
↳ Counterbalancing - giving participants different orders of the same conditions to control for order effects (ex: half the students use pen → laptop, the other half use laptop → pen)
↳ Paired designs are more sensitive than independent groups designs
Week Seven - Lecture Two
Where does variability come from? (independent groups)
Random Sampling Variability (error) - random “noise”
Difference among population means - true group differences
Between-group variability = Random Sampling Variability + Population Mean variability
Within-group variability = Random Sampling Variability
ANOVA - between subjects
SS - Sum of Squares; measures total amount of variability in a dataset
MS - Mean square; the average variability (SS/df)
→ For a one-way ANOVA, the effect size is measured by: n2 = (SSbetween) / (SStotal)
** SS total = SSbetween + SSwithin
F-Test - (Between Subject Variability) / (Within-Subject Variability)
↳ Assuming the null hypothesis is true, between-subject variability and within-subject variability will be roughly equal
F-Distribution - between subjects
↳ Unlike the t-distribution, which has only one df value, the F-distribution has two df values
↳ k = number of groups (conditions) ; n = total number of participants
↳ df 1: k-1
↳ df 2: n-k
Assumptions for a One-Way (between-subjects) ANOVA
Homoegenity of Variance -
Each group has a similar amount of variability
If one is much more variable than another it can mess with results and make it harder to tell if differences in means are real
Normality of Residuals
Residuals are the difference between an individual score and the group mean
ANOVA assumes these are normal, if skewed the results give misleading p-values
Random Assignment
Participants should be randomly assigned to groups to avoid bias influencing results
We want results to be due to independent variable not differences between people
Repeated Measures ANOVA
↳ df 1: (k-1)
↳ df 2; (k-1)(n-1)
**For repeated measures ANOVAs there is one additional assumption:
Sphericity Assumption: The variability of the difference scores between each pair of conditions must be equal
→ We test this with Mauchly’s Test, and if significant sphercity is violated
→ If violated we adjust degrees of freedom to correction p-value (using greenhouse-geisser or huynh-feldt)
→ For a one way RM-ANOVA, the effect size is measured by:
n2 = SSbetween / SSbetween + SSerror
BENCHMARKS FOR n2:
SMALL - .01
MEDIUM - .06
LARGE - .14
Week Eight - Lecture One
Factorial Designs
↳ A study (either between-subject, within-subject, or both) in which there is more than one independent variable, and they are crossed with each other
Ex: The researcher wants to examine how the frequency (high/low) and luminance (bright/dim) of each word impacts RT
Null Hypotheses for Factorial Designs:
→ The population means for each condition are equal
→ There is no difference between population means for each condition, at each level
Cell Means: The mean associated with a specific combination of levels
Marginal Means: The average across levels of IV
Example - Study testing studying (using pen, using laptop) and (quiet, noisy)
Cell means: Pen + Quiet = 80 or Laptop + Noise = 65
Marginal Means: Pen (Average of quiet and noise) = 80+75/2 = 77.5
F-Test for Factorials: Study of Luminance (dim/bright) and Frequency (low/high)
↳ Flum = MSluminance / MSwithin
↳ Ffreq = MSfrequency / MSwithin
↳ Ffrequ*lum = MSfrequency*luminance / MSwithin
Significant Main Effect:
↳ An IV has an effect regardless of what other factors it is associated with
↳ For the study example: Pen would have an effect whether paired with a noisy or quiet room
Significant Interactions:
↳ When the effect of one IV changed depending on the level of another IV
↳ In the study example, during quiet study pen leads to higher scores, but in noisy study laptop will lead to higher scores
↳ If there are non-parallel lines, it suggests a possible interaction
Simple Main Effects:
↳ Examines effect of one IV at a certain level of another IV
↳ Helps determine where differences actually occur
Ex: Studying Pen vs. Laptop just in quiet study
Measures of Effect Size:
↳ Effect size is measured by partial eta-squared (n2)
↳ (SSiv) / (SSiv) + (SSerror)
↳ Essentially tells us, out of the total variability in scores for this effect, what proportion is explained by the IV/interaction (0 → the IV explains NONE of the variability, 1 → the IV explains ALL the variability)
FACTORIAL ANOVA : step by step guide
Start with interactions:
→ If there is a significant interaction, you look at simple main effects (testing one IV against two levels of other IV)
→ If not significant, then proceed to significant main effects
Check Main Effects:
→ A main effect tells you if an IV has an overall effect, ignoring the other IV
→ If significant:
IV of 2 levels: You are done, there are only two levels so you know which is higher
IV > 2 levels: You must do post-hoc tests to determine which specific levels differ from each other
→ If not significant, then IV likely has no effect, so no further tests are needed
Week Nine - Lecture One
2x3 Factorial Design: example drug use (drug x, y, z) and luminance (bright, dim)
Two IVs:
IV1 - Two levels (bright/dim)
IV2 - Three levels (drug x, y, z)
Follow same guide as 2x2 factorial ANOVA:
→ Remember though, if IV has 3 levels and there is a main effect, post-hoc tests are needed to see which levels differ