Introduction to Data Collection

Importance of Data

  • Clinical Scientist Model: The practice of using a scientific approach to delivering clinical services

  • Evidence Based Practice: The integration of best available research with clinical expertise while taking into account client characteristics

  • Pseudoscience: treatments that are presented as if they have a scientific basis, but have not or cannot be supported scientifically

  • ABA as Applied Science

    • Generalizable

      • skills exist in other contexts

    • Effective

      • ensured to achieve goals

    • Technological

      • described enough for replication

    • Applied

      • socially significant behaviors are selected

    • Conceptually systematic

      • consistent with supporting literature

    • Analytic

      • data-based

    • Behavioral

      • observable and measurable

  • ABA vs Pseudoscience

    • unbelievably generalizable

    • effectiveness unproven

    • proprietary technology

    • personally applied

    • conceptually unsystematic

    • nonanalystic

    • nonbehavioral


Types of Data We Collect

  • Types of Data

    • Continuous: every possible observations

    • Discontinuous: sample observation

  • Continuous data

    • frequency

    • rate

    • duration

    • latency

    • inter-response time

  • Discontinuous data

    • partial interval

    • whole interval

    • momentary time sample

    • planned activity check


Reliability

  • Reliability: the degree to which data is replicable

  • Interobserver Agreement: percent agreement between the data collected simultaneously by two independent observers recording the same observation

  • Importance of reliability

    • new data collectors are properly trained

    • guards against observer drift

    • ensures behavior definitions are tight

    • ensures data reflects reality

  • Ways of Calculation IOA

    • Counted Data

      • Total count IOA
        mean count IOA

      • exact count IOA

      • trial by trial IOA

    • discontinuous data

      • interval by interval

      • scored interval

      • unscored interval

    • duration data

      • total duration

      • mean duration per occurence

  • Total Count

    • smaller count/larger count x 100

    • simple, quick calculation

    • little precision

  • Mean Count

    • (interval 1 IOA + Interval n IOA)/ n intervals x 100

    • better precision than total count IOA

    • more steps involved

  • Exact Count

    • (# of intervals 100% IOA/n intervals) x 100

    • easy to calculate and highly precise

    • few situations require this level of precision; provides and underestimate socially important reliability in many cases

  • Interval by Interval

    • # agreed trials / n trials x 100

    • easy, broadly informative calculation

    • limited range of usefulness

  • Scored or Unscored

    • (# of agreed intervals / # of intervals) x 100

    • gives more precise information of where disagreement is occurring

    • annoying setup

  • Total Duration

    • (shorter duration/longer duration) x 100

    • simple, quick calculatiom

    • little precision

  • Mean Duration

    • (Duration R1 + Duration RN… / n responses) x 100

    • Better precision than total duration

    • more steps involved


Validity

  • Validity: The degree to which data accurately reflect the phenomenon they are reported to describe

  • Validity factors

    • accuracy

      • obtained value reflective of the true value

    • representativeness

      • represents the behavior across time. conditions and settings

    • relevance

      • data captures the most relevant of the dimensions of the target behavior

    • significance

      • measured behavior is socially significant

  • Relationship between validity and reliability

    • valid data must be reliable

    • reliable data is not necessarily valid