Using SAFMEDS to Increase Undergraduate Success with Statistical Concepts

Numerical Competence and the Statistics Challenge

• The ability to interpret mathematical and statistical information is considered an essential functional skill in modern, information-laden society, as figures are routinely used in media to influence perception (Tufte, 1983).

• Accurate interpretation is vital for making informed decisions, particularly in workplace settings (Huff, 1993).

• General numeracy levels in the UK are currently considered poor:

  • A survey in England identified approximately $15$ million adults possessing only entry-level numeracy skills (Grinyer, 2005).

  • A survey for the Welsh Assembly Government ($2005$) found that $53\%$ of adults did not possess level one numeracy skills.

  • Level one numeracy is defined as equating to the lowest achievable grade mark without failing a General Certificate of Secondary Education (GCSE) examination (approximately below the $G$ grade standard).

• Psychology undergraduate students show evidence of deteriorating mathematical abilities, as highlighted by a comparison of cohorts entering departments a decade apart (Mulhern \& Wylie, 2004).

• Science undergraduates face a mandate to be competent in statistics to conduct research effectively and evaluate the research of others (Everson, Zieffler, \& Garfield, 2008).

• Statistics is often difficult for students; the complexity of learning the vocabulary and concepts has been likened to learning a second language (Lalonde \& Gardner, 1993; Lazar, 1990).

Behavioral Fluency and the RESA Framework

• Accomplished performance, or mastery, is characterized as effortless, automatic, and accurate responding, often referred to as automaticity (Bloom, 1986; Binder, 1996).

• Fluency in basic skill elements is considered a prerequisite for learning higher-order, complex compound skills (Kubina \& Morrison, 2000).

• Sequences of learning require that each element be mastered (e.g., mathematical operations) before proceeding to the next (Stein, Silbert, \& Carnine, 1997).

• Cumulative Dysfluency: A term coined by Binder ($1996$) to describe a deficit in skill elements that prevents a learner from progressing through a curriculum.

• Fluency Aims: Specific performance targets or rates of accurate responding documented for academic skills (Kubina, 2002). For reading, the aim is between $200$ and $250$ words of prose per minute (Hughes, et al., 2007).

• RESA Outcomes: An acronym representing the benefits of fluent performance (Fabrizio, 2004; Johnson \& Layng, 1992):

  • Retention: The ability to perform a skill fluently after long periods without practice.

  • Endurance: The ability to perform a skill for extended periods of time.

  • Stability: The ability to perform a skill in the presence of distractions.

  • Application: The ability to combine previously learned sub-skills to perform novel, complex compound skills.

Principles of Effective Instruction and Precision Teaching

• Conners, McCown, \& Roskos-Ewoldsen ($1998$) identify four main problems in teaching statistics:

  1. Variability in individual performance.

  2. Student motivation.

  3. Statistics anxiety.

  4. Making learning last.

• Effective educational practices (Frederick \& Hummel, 2004) require:

  • Clearly stated behavioral objectives.

  • Accurate, competent models.

  • Frequent opportunities for active responding.

  • Immediate feedback on accuracy.

  • Self-pacing.

  • Teaching to mastery.

  • Reinforcement of accurate responses.

  • Frequent and direct measurement tied to behavioral objectives to inform instructional decisions.

• Precision Teaching (PT): An effective intervention with four decades of evidentiary support (Beck \& Clement, 1991). Learners often use short, timed practice sprints to collect data on correct and incorrect responses per minute (Learning Opportunities; LO).

• Standard Celeration Chart (SCC): A tool used to plot data pairs (corrects and errors). It uses a multiply-divide (logarithmic) scale (Lindsley, 1995; White \& Neely, 2004). The SCC allows teachers and learners to observe learning pictures and make data-driven decisions.

• SAFMEDS: An acronym for "Say All Fast Minute Every Day Shuffled." Developed by Graf \& Lindsley ($2002$), these are practice and assessment cards used to build fluency in definitions and basic concepts.

Study Methodology and Participant Selection

• Initial Pool: Approximately $340$ first-year psychology students.

• Selection Criteria: Students scoring at or below the $50^{\text{th}}$ percentile on their first weekly class test were invited to participate.

• Final Sample ($n=55$): Randomized allocation resulted in $24$ students in the PT intervention group and $31$ students in the Treatment as Usual (TAU) group.

• Age Range: Participants were aged $18$ to $30$ years and earned course credits for participation.

• Stimuli: A 50-item multiple-choice pre-test covering a semester's worth of content was used. Questions were taken from Gravetter \& Wallnau ($2007$).

• SAFMEDS Material: Two packs of $80$ cards each were created from the Gravetter \& Wallnau ($2004$) study guide.

  • Pack 1: Covered content for weeks $3$ to $5$.

  • Pack 2: Covered content for weeks $6$ to $11$.

• Design: A mixed design with one between-group factor (PT vs. TAU) and one repeated measures factor (Pre-test vs. Post-test).

Experimental Procedures and Materials

• Pre-test: Both groups took a 50-item pre-test with a $45$-minute time limit.

• TAU Group: Continued with standard lectures, one-hour weekly revision sessions, and normal preparation.

• PT Group: Attended standard lectures but also received training in PT methods and the SCC rationale.

• PT Daily Routine:

  • Conduct 1-minute timings $3 \times$ per day ($\approx 6$ minutes total).

  • Plot the best score of the day on an SCC.

  • Meet twice weekly with student proctors to verify data and adjust learning strategies.

• SAFMEDS Procedure: Each card featured a statistical definition with a missing term on the front and the answer on the reverse.

  • Example 1: Front: "standard deviation is the _____ of the variance"; Back: "square root".

  • Example 2: Front: "_____ distribution"; Back: "bimodal".

  • Learners say the answer aloud, check the back, and sort cards into "correct" or "learning opportunity" (error) piles.

• Fluency Aim: The goal set was $60\text{--}40$ correct per minute with no more than $2$ errors.

• Interventions: Procedures used if progress stalled included "slice-backs" in card count, changing timing periods, or error correction (untimed pack reviews where every card is answered correctly before daily timing).

Analysis of Results and Statistical Significance

• ANCOVA Results: Post-test mean percentage correct scores showed a significant difference favoring the PT group: $F(1, 53) = 5.23$, $p = 0.026$, $d = 0.62$ (a medium effect size).

• Weekly Tests: The PT group consistently achieved higher mean scores on weekly open-book tests from weeks $5$ to $11$.

• Week 5 Performance: After only one week of using SAFMEDS Pack 1, the PT group outperformed the TAU group by more than $10\%$.

• Individual Improvement (Pre- to Post-test):

  • PT group range of improvement: $0$ to $42$.

  • TAU group range of improvement: $-8$ to $28$.

  • $33.33\%$ ($8$ participants) from the PT group increased scores by more than $2$ standard deviations; $4$ students in the PT group increased scores by $\ge 3$ standard deviations.

  • Only $16\%$ ($5$ participants) of the TAU group achieved a >2 standard deviation increase.

Discussion, Individual Learning Pictures, and Future Directions

• The PT intervention resulted in an improvement equivalent to almost one entire degree class score (e.g., from a $C-$ to a $B-$).

• SCC Case Study: One participant working with Pack 1 showed a "crossover jaws" learning picture where errors started high and corrects started low, then crossed as corrects increased and errors decreased.

• The "Jump Up" Effect: When Pack 2 was introduced, error rates (LO) for Pack 1 jumped up. This was interpreted as the learner becoming more comfortable and attempting a higher volume of cards ($30$ per minute vs. early $15$ per minute), which increased speed but temporarily increased total errors.

• Limitations: Research relied on participant self-reporting for home timings. Lecture attendance and general study time for TAU were not monitored.

• Future Research Directions:

  • Internet-based SAFMEDS systems to track latency and automatically adjust card presentation based on mastery.

  • Systematic exploration of the relationship between response latency and RESA outcomes.

  • Use of comparison groups that also use flashcards but without PT/timing/charting components.

  • Monitoring whether PT reduces statistics anxiety levels.

• The study concludes that PT and SAFMEDS are low-cost, easily implemented methods that effectively enhance learning and can help diminish skill deficits across various curricula.