Blackman, A. L., DiGennaro Reed, F. D., Erath, T. G., & Henley, A. J. (2022)

Introduction & Background

• Rapid expansion of credentialed behavior‐analytic professionals over the past decade
  • $BCBA$ and $BCaBA$ numbers doubled between $2015$ and $2021$
  • $RBT$ population grew seven-fold during same period (total $172,162$ certificants in $2021$)
  • Increasing service demand → heightened importance of high-quality staff training, performance management (PM), and supervision
• BACB credentials set only entry-level competency standards (coursework + exam)
  • Passing multiple-choice exams does not guarantee expertise → ongoing professional development is ethically required
• DiGennaro Reed & Henley $2015$ survey exposed weak training/PM practices (e.g., $45\%$ no preservice training, $60\%$ no performance observation)
• Subsequent changes
  • Introduction of $RBT$ credential $2014$, monthly $5\%$ supervision requirement
  • New BACB supervision requirements (e.g., first-year consultation for new $BCBA$s)
  • Expanded literature on best-practice supervision (e.g., Sellers et al., Valentino et al.)

Study Purpose

• Replicate & extend DiGennaro Reed & Henley $2015$ to capture current organizational practices
• Focus areas:
  • Initial (preservice) staff training
  • Ongoing (in-service) training
  • Performance management mechanisms
  • Provision & quality of supervisory-skills training
• Compare findings to $2015$ baseline and highlight improvements, regressions, new concerns

Methodology

• Design: Anonymous cross-sectional online survey (Qualtrics)
• Recruitment channels
  • BACB email blast to opt-in certificants
  • $14$ behavior-analytic Facebook/SM groups (e.g., “Students of ABA”)
• Inclusion: Complete $\ge 30\%$ of items
• Data collection window: $5\,Dec\,2019$ – $7\,Jan\,2020$
• Final sample: $N = 578$ (out of $629$ starts)
  • Unable to compute response rate (unknown denominator)

Instrument Structure

1. Demographics (10 items)
2. Initial orientation/preservice training (7 items)
3. Ongoing/in-service training (5 items)
4. Supervision practices & training (4 main + follow-ups) – completed only by supervisors

• Minor modifications vs. $2015$ survey: removed incentive questions, added RBT-specific supervision queries

Participant Demographics

• Gender: $85.3\%$ female, $13.3\%$ male, remaining nonbinary/other
• Mean age $=36.2$ years (range $19–76$)
• Ethnicity: $86.3\%$ White; $8.3\%$ Hispanic origin
• Highest degree
  • Master’s $62.1\%$, Doctorate $15.4\%$, Bachelor $17.0\%$
• Field of study: ABA $51\%$, Special Ed $12.7\%$, General Psych $10.1\%$
• Credentials held
  • $BCBA$ $55.5\%$, $RBT$ $22.7\%$, $BCBA! -D$ $11.4\%$, $BCaBA$ $1.9\%$, others/aspiring $8.5\%$
• Tenure
  • $73.9\%$ certified ≤ $6$ years
  • $78.5\%$ employed at current job ≤ $6$ years
• Work settings (top): private center $30.5\%$, client homes $17.7\%$, public schools $14.4\%$
• Primary roles: clinician/supervisor $28.7\%$, direct care $22.8\%$, admin/manager $14.2\%$

Initial / Preservice Training Findings

• Availability
  • $66.3\%$ overall received preservice training
  • $BCBA$ $59\%$ | $BCaBA$ $92\%$ | $RBT$ $89\%$
• Duration reported (modal)
  • $1–3$ days $28.5\%$ (esp. BCBA, RBT)
  • $6–10$ days $22.8\%$ (esp. BCaBA)
  • >$2$ weeks $17.5\%$ (wide variability for RBTs)
• Delivery modalities (multiple possible)
  • Face-to-face $94\%$
  • Online synchronous $29.1\%$ (increase vs. $2015$)
  • Online asynchronous $7.1\%$
• Training components used
  • Verbal instructions $82.7\%$
  • Written instructions $66.2\%$
  • Modeling $63.1\%$
  • Lecture $58.1\%$
  • Peer shadowing $57.9\%$ (popular yet under-researched)
  • Role-play $49.2\%$
  • Practice with clients $40.6\%$
  • Performance feedback $54.2\%$ (mostly verbal, delivered throughout)
• Mastery criteria presence
  • Quizzes $85.5\%$ had mastery cut-off
  • Role-play $52.5\%$ | In-vivo practice $51.3\%$
• Perceived adequacy
  • “Prepared me” = $55.9\%$ overall (BCBA $53\%$, BCaBA $50\%$, RBT $70\%$)
• Changes vs. $2015$: Higher coverage of training, more online use, but lower self-rated preparedness

Ongoing / In-Service Training Findings

• Availability: $80\%$ overall
  • $BCBA$ $77\%$ | $BCaBA$ $91\%$ | $RBT$ $89\%$
• Common formats
  • On-site workshops/lectures $67\%$ (often monthly $43.1\%$)
  • Supervisor observation $67\%$ (weekly $36.6\%$ typical)
  • Performance feedback $69.7\%$ (verbal $92.5\%$, supervisor-delivered $86.8\%$)
  • Online trainings/webinars $49.1\%$ (up sharply)
• Relevance: $68.3\%$ judged topics directly relevant; $29.4\%$ “sometimes” relevant
• Performance management: More frequent workplace observations than $2015$ (positive trend)

Supervisory Training & Practices

• Supervisory responsibilities: $60.9\%$ of respondents
  • Staff supervised: mostly <$15$ persons ( $84.1\%$ )
  • Populations: RBTs $42.8\%$, non-certified DSPs $39.3\%$
• Training on supervision
  • Only $43.1\%$ received any formal supervisory-skills training
  • Of these, >95\% said it “fully or somewhat” prepared them
  • $56.9\%$ supervise without formal preparation → area of concern
• Quality assurance
  • Employer policy to monitor supervision: $46.3\%$ yes / $46.3\%$ no
  • Opportunities for supervisee feedback: $64.9\%$ yes (mostly supervisor solicitation $71.6\%$; anonymous forms $43.2\%$)
• Observation of supervisor performance: $44.4\%$ “never” observed while supervising

Key Comparisons with 2015 Baseline

• Demographic shift: more RBTs, fewer master’s-level proportions, more direct-care roles
• Improvements
  • Higher % receiving preservice (+ ongoing) training
  • Greater use of role-play, mastery criteria, and PM observations
  • Expansion of online/synchronous delivery
• Continuing problems
  • Preservice training still relies mainly on instructions/modeling; practice & feedback under-utilized
  • BCBA group receives least initial + ongoing training
  • Self-reported preparedness fell relative to $2015$
  • Majority of supervisors lack formal supervisory-skills training despite BACB ethical requirements

Practical Implications & Recommendations

• For Organizations
  • Institute performance-based preservice programs: include practice, feedback, mastery targets for all credential tiers
  • Formalize peer-shadow protocols → train “peer mentors,” set standardized observation checklists, collect effectiveness data
  • Track training impact (e.g., pre-/post skill probes, post-deployment performance data)
  • Ensure ongoing training for BCBAs: advanced content (cultural humility, compassionate care, OBM, etc.)
  • Adopt evidence-based PM: routine observations, feedback, pay-for-performance, CE support (cf. Novak et al. $2019$)
  • Provide structured supervision curricula (Reid et al. $2012$; LeBlanc et al. $2020a$; Sellers et al. $2016b$)
  • Implement supervision quality audits and bidirectional feedback systems; comply with BACB Ethics Code ## $\text{(2.0, 5.0)}$
• For Funders / Regulators
  • Tie reimbursement or accreditation to evidence-based staff training & PM standards
• For Researchers
  • Evaluate efficacy of peer-shadowing & conference/webinar CE on actual job performance
  • Identify optimal blends of asynchronous, synchronous, and in-person modalities
  • Investigate social-support facets of online training and their effect on burnout

Study Limitations (Author-Acknowledged)

• Self-report bias; no direct verification of organizational practices or performance data
• Possible clustering (multiple respondents from same agency) could skew frequencies
• Small $BCaBA$ sample ($n=11$) limits inferences
• Tenure variability: some new hires may lack full exposure to existing programs

Connections to Ethics & Broader Field

• BACB Ethics Code ## requires evaluating effects of supervision; results show compliance gaps
• High-quality staff training directly influences client outcomes, social validity, and professional reputation of ABA
• Burnout findings (Plantiveau et al. $2018$): insufficient supervision & training ↔ decreased job satisfaction → turnover risk
• Emphasis on compassionate & culturally humble practice aligns with emerging ethical imperatives (LeBlanc et al. $2020b$; Beaulieu et al. $2019$)

Key Numbers & Quick Reference

• Total respondents $N = 578$
• Preservice training received $66.3\%$ | Ongoing training $80\%$
• Supervisors without supervision training $=56.9\%$
• BCBA preservice coverage $59\%$ vs. RBT $89\%$
• Preparedness ratings: only $55.9\%$ felt ready post-training