W4 - Measurement, Distributions, and Percentiles - Comprehensive Notes
Acknowledgement and Course Context
Welcome note and course focus: measurement, frequency distributions, and percentiles; gentle introduction to numbers.
Mid-semester exam scope: weeks 1–4; scheduled examSaturday, September 6 (announced on Blackboard).
Course trajectory: earlier weeks covered scientific process, study design, and questions in psychology; this week moves to data after collecting numbers.
Practical relevance: data cleaning, exploration, and plotting are essential across assignments, in honors year, and in the research process.
Measurement, Constructs, and the Philosophy of Measurement
Measurement goal: assign numbers to objects/observations according to consistent rules (operational definitions).
Constructs in psychology: psychological phenomena like anxiety, memory that are not directly observable but are labeled and studied.
Operational definition: boundaries/criteria to determine whether a phenomenon (construct) is present in a measured instance (e.g., infant imitation). Researchers may disagree; scientific discourse can refine definitions over time.
Observable phenomena and empiricism: measurement relies on observable, checkable, verifiable evidence shared openly for replication.
Scientific disagreement as progress: debate over definitions/methods pushes for better processes.
Variables: Types, Qualities, and Scales
Variable: a characteristic of interest for each individual in a population/sample (e.g., memory capacity, anxiety).
Qualitative vs. quantitative variables:
Qualitative: categories/labels (e.g., gender, eye color, political affiliation); not meaningful to compute averages.
Quantitative: numeric measures (e.g., height, weight, income); meaningful to apply statistics.
Coding and measurement rules:
Numbers can be used as labels (e.g., 0/1 coding for deceased/alive) but not all label-numbers support arithmetic operations.
Types of variables (overview, basic):
Discrete: whole-number values (no meaningful halves). Example: number of cars observed in a period.
Dichotomous: two possible values within discrete (e.g., yes/no; male/female; correct/incorrect).
Continuous: any value within a range (e.g., height, volume).
Measurement scales (order of sophistication):
Nominal: labels without meaningful order; e.g., color categories, political parties, jersey numbers.
Ordinal: ordered categories where order matters but intervals are not necessarily equal.
Interval: ordered with meaningful differences between values, but no meaningful zero. Example discussed: IQ differences; temperature scales like Celsius.
Ratio: interval properties plus a meaningful zero, allowing ratios (e.g., height, weight, Kelvin temperature).
Examples and nuances:
IQ: ordinal → interval when actual scores provided; distance between scores meaningful.
Temperature: Celsius is interval (differences meaningful) but lacks a true zero; Kelvin is ratio (has meaningful zero).
Age: often treated as ratio (meaningful zero) in many contexts; sometimes discussed as interval in teaching contexts.
Implications of scale choice for analysis: the chosen scale constrains which statistics and claims are valid.
Practical examples in measurement:
Eye color as nominal; cannot average eye color.
Height as ratio; allows means, proportions, comparisons like “twice as tall.”
How to report numbers: use of consistent labels and units; interpretability depends on scale properties.
Reliability and Validity of Measures
Reliability: consistency of a measure across time or raters.
Test-retest reliability: administering the same test twice should yield similar scores if the underlying trait is unchanged.
In practice, perfect identical scores are unrealistic due to day-to-day variation (e.g., sleep, mood).
Reliability is quantified via correlation between scores across occasions:
If scores on Test 1 and Test 2 are highly correlated, reliability is high.
Inter-rater reliability: when multiple raters judge the same thing (e.g., video ratings), their scores should be correlated.
Typical adequacy: correlations around 0.60 or higher are considered acceptable for reliability in many contexts.
Example: alpha waves as a biological fingerprint show very high test-retest reliability over months (almost identical scores).
Validity: whether a measure actually assesses the intended construct.
Internal validity: the extent to which observed effects are due to the manipulated variables, not confounds.
External validity: generalizability of results beyond the lab to real-world settings (issues with WEIRD samples: Western, Educated, Industrialized, Rich, Democratic).
Construct validity: whether the measure truly taps the theoretical construct (e.g., Beck Depression Inventory potentially overlapping with anxiety items; concerns about how well items map to depression construct).
Content/face validity: whether the measure appears to assess the intended construct on the surface (e.g., mental math tests appearing to measure math ability; head circumference appearing to measure head size, not intelligence).
Predictive validity: the extent to which scores on a measure predict related outcomes (e.g., ATAR predicting university performance).
Other validity considerations:
Construct validity and evolving measures: early measures may drift as constructs are better understood; poor initial alignment may be revised.
Content/face validity distinctions: a measure can be reliable but have low face validity if it doesn’t intuitively fit the construct.
Reliability vs validity relationship: a measure can be reliable but not valid; it must measure what it intends to measure to be useful.
Pilot Testing, Range Effects, and Study Design Considerations
Pilot testing: iterative testing of experimental design and stimuli to ensure the measurement range is appropriate.
Goals: avoid floor effects (too hard) and ceiling effects (too easy); ensure middle-range performance to observe differences.
Real-world example: quick demonstration with speed of stimulus presentation; initial results suggested adjustments to avoid near-zero performance.
Range effects and measurement quality:
Ceiling effect: all participants perform near the top; little room for differentiation.
Floor effect: all participants perform near the bottom; little room for differentiation.
Ideal measures sit in a middle range to maximize sensitivity to group differences.
Pilot testing as a standard in research: many published studies include extensive pilot work not visible in the final paper.
Study design considerations discussed earlier in the course:
Types of studies: experimental, randomized controlled trials; observational, quasi-experimental, correlational.
Randomization and control groups as tools to manage confounds.
Independent groups design vs. repeated measures design; counterbalancing as a method to balance potential confounds.
Construct-focused design notes: importance of naming and constructing meaningful constructs before measurement.
Data Presentation, Exploration, and Cleaning
Purpose of data presentation: to tell a clear story about results using figures and tables rather than lengthy narrative only.
Data are often messy: human data can include errors, non-sensical responses, and noise; cleaning is essential before analysis.
Data cleaning and exploration steps:
Inspect raw data to identify values outside plausible ranges (e.g., 0–10 scales with a value of 20).
Look for transcription or entry errors (e.g., too-high values in a given scale).
Clean data and summarize before performing analyses.
Data organization example: raw data matrix (100 students × 10 true/false questions) vs. summarized representations.
Summary representations help reveal patterns quickly:
Frequency tables: list all possible scores and the count of observations per score.
Frequency of 0–10 scores example: helps identify most common scores and check data integrity.
Frequency tables vs. variability in data:
With many possible scores, frequency tables become unwieldy; interval-based bins improve readability.
Rule of thumb: 10–20 intervals (bins) balance granularity and interpretability; 15 bins often cited as a good middle ground.
Interpreting frequency data:
Relative frequency: proportion of observations in each bin:
Cumulative frequency (CF): total observations with scores at or below a given bin.
Percentiles: boundaries where a given percentage of scores fall below that value.
Practical examples: weights of 72 male students; intervals like 60–64, 65–69, etc.; note about inclusive/exclusive bin definitions to avoid overlaps.
Why include empty/zero-edge bins: to enable certain plots (e.g., frequency polygons) that require zero values at the ends.
Frequency polygons and alternative plots:
Frequency polygon visually connects bin midpoints to show distribution shape.
Bar graphs for nominal data; histograms for continuous data with touching bars to show continuity.
Box-and-whisker plots provide information about median, interquartile range, and extremes.
Bar graphs vs. histograms:
Bar graphs: for qualitative (nominal) data; bars not touching; order is flexible to aid readability.
Histograms: for continuous data; bars touch to indicate continuity between bins; bin intervals matter.
Frequency polygons for multiple groups:
Example with male actual weight vs. male ideal weight; female weights and ideal weights plotted to compare distributions.
Telling a story with graphs:
Well-chosen figures reveal patterns and differences (e.g., male vs. female weight patterns and ideal vs. actual weights).
Graphs should be designed to convey a clear message, guiding interpretation.
Summary points for data presentation:
Sift, clean, and present data so a reader can understand at a glance.
Good figures prepare the data for inferential tests (e.g., verifying assumptions, handling missing data, removing outliers).
Choose graph types that best fit the data type (qualitative vs. quantitative) and the story you want to tell.
Use appropriate intervals (bins) when constructing histograms/frequency polygons.
Percentiles, Cumulative Frequencies, and Practical Calculations
Percentile concept: the value below which a specified percentage of scores fall.
90th percentile: 90% of scores are below this value.
Percentiles are computed by ranking scores and locating the boundary that separates the specified percentage of data.
Relative frequency vs. cumulative frequency:
Relative frequency: proportion of the total represented by a score/bin:
Cumulative frequency (CF): sum of frequencies for all scores up to a given point.
Percentile calculation method:
Percentile rank =
To find the percentile of a given score, determine CF up to that score and divide by N, then multiply by 100.
Example walkthrough: with a table of scores and frequencies, CF is calculated by summing frequencies up to the target score; percentile = (CF / N) × 100.
Inverting percentile calculations:
To find the score corresponding to a given percentile p, compute CF = (p/100) × N, then locate the score bin whose cumulative frequency reaches CF.
Worked example (class scores):
Suppose a small table with scores and frequencies; N = 20; to find the 35th percentile: compute CF = 0.35 × 20 = 7; find the score with CF of 7 (e.g., a score of 23). Therefore, 35th percentile corresponds to score 23.
Interpretation: a student scoring 23 did better than 35% of the class.
More advanced example: hours of TV watched by 259 students (data from a lecture):
Distribution across categories (0–1, 2–3, 4–5, etc.) with cumulative frequencies calculated up to seven hours.
To find the percentile for seven hours, compute CF up to 7 hours and divide by 259, then multiply by 100; here, around the 63rd percentile.
Frequency polygon interpretation example:
Shade region left of a percentile boundary to visualize the proportion of data below that boundary (e.g., 63% area under the curve to the left of 7 hours).
Modern data practices: most computing of percentiles and other statistics is done with software, but understanding the underlying calculations is essential for intuition and debugging.
Summary of percentiles in reporting:
Percentiles provide meaningful benchmarks (e.g., “above 80% of the class”).
Use cum freq and N carefully to avoid misinterpretation; ensure you read the correct cell when using rearranged equations.
Graph Types and Data Storytelling: Choosing the Right Display
Bar graphs (qualitative data):
Display counts per category; y-axis scale should reflect observed counts; bars should not touch to emphasize categorical separation.
Ordering the categories can help readers see patterns; the order is not inherently meaningful for nominal data but can aid interpretation.
Histograms (quantitative data):
Bars touch to indicate a continuum between bins; choose bin width carefully to reveal distribution shape without over-smoothing or over-fragmentation.
Box-and-whisker plots: quick view of distribution shape, median, interquartile range, and extremes.
Frequency polygons: smooth representation of distributions by connecting bin midpoints; useful for comparing distributions (e.g., groups vs. groups).
Practical storytelling with plots:
Use a graph to illustrate differences (e.g., male actual vs. ideal weight distributions) and to compare groups (e.g., male vs. female weight patterns).
Good plots support the narrative of your results and help convey your claims effectively.
Practical Advice for Exam Preparation and Next Steps
Data workflow in research:
Design measure with appropriate scale; pilot test to refine range and avoid floor/ceiling effects.
Collect data, then clean and explore before formal analysis.
Create figures that tell a story; choose graphs that fit the data type and the message.
Prepare for inferential statistics by ensuring data meet assumptions (normality, etc.).
Mathematical basics to brush up for next week:
Σ notation:
$$\