Measurement and Graphing

Operational Definition

Clear, concise, and specific description of a concept or variable in terms of the operations or procedures used to measure or identify it.

Can be used to translate abstract or subjective terms into measurable behaviors that can be observed and recorded.

Observable

It specifies behaviors that can be seen or heard by others, not internal states that are subject to interpretation.

Measurable

The definition allows for the behavior to be counted or quantified in some way (e.g., frequency, duration, intensity).

Objective

It is stated in such a way that two independent observers can recognize and measure the same behavior without subjective judgment.

Clear and Unambiguous

It avoids vague language and provides enough detail so that anyone reading the definition would understand exactly what behavior is being referred to.

Complete

The definition should include all instances in which the behavior occurs, as well as instances in which the behavior does not occur (to prevent false positives or negatives).

Individualized

The particular forms of a behavior that one individual display will likely be different from those of another.

Observer Drift

This occurs when observers unknowingly alter the way they measure behavior over time, drifting away from the operational definition.

Observer Bias

When an observer’s expectations or personal beliefs influence the data they collect, resulting in a skewed representation of the behavior

Inadequate Observer Training

Without proper training, observers may not accurately follow the defined protocols for data collection, leading to inconsistencies and errors

Poor Operational Definitions

If the behavior being measured is not clearly defined, different observers may interpret and record it differently

Observer Fatigue

Over long observation periods, an observer may become tired, leading to lapses in attention and inaccurate data recording

Sampling Bias

If data is only collected during certain times or in specific situations, it may not be representative of the behavior as it occurs naturally.

Measurement Bias

This can occur when the measurement tools or methods used are not consistent with the behavior being observed (e.g., using an interval recording when a frequency count would be more appropriate)

Reactivity

Sometimes, simply the presence of an observer can change the behavior of those being observed, which can skew the data

Data Entry Errors

Mistakes made when inputting data into databases or datasheets can lead to analyses based on incorrect information

Insufficient Inter-Observer Agreement (IOA)

Without regularly checking that different observers record the same behavior in the same way, data reliability can be compromised

Validity

Relevance to the Behavior’s Dimension, and Representativeness of Behavior Occurrence

Wrong Dimension

When the chosen dimension of measurement does not align with the actual characteristics of the behavior

Artifact

A distortion or misrepresentation in the data leading to a potentially invalid measurement

Indirect Measurement

Involves using a substitute or proxy dor directly observing the behavior

Accuracy

Assesses how closely the observed value aligns with the true value of whatever is being measured

Reliability

Consistency of measurement results over time and across different conditions

Poorly Designed Measurement Systems

Refers to measurement tools or procedures that are excessively complicated or burdensome

Continuous measurement

Refers to the ongoing collection of data over a period of time, capturing the frequency, rate, duration, and latency of events or behaviors.

Frequency

Measures how often a particular event or behavior occurs within a specified time frame.

This is a simple count of occurrences, valuable for understanding the prevalence or commonality of the observed phenomena.

Rate

Overall measure of the behavior’s occurrence per unit of time

This allows for the comparison of data across different time periods or among various studies where the durations of observation may differ.

Ex: Rate of hitting behavior per minute = (number of hits) / (Observation time in minutes)

15 hits / 30 minutes = 0.5 hits per minute

Duration

Records the length of time a behavior occurs/lasts

Captures how long it persists each time it occurs.

*When the behavior is occuring

Inter-Response Time (IRT)

The time between responses.

To take IRT, you begin the stopwatch when a behavior ends and stop the stopwatch when the next behavior begins.

*When the behavior is not occurring

Latency

The amount of time that elapses between the presentation of a prompt/instruction and the initiation of the corresponding behavior

Start stopwatch at the end of a prompt, end it when the behavior begins

*The time it takes for a behavior to start

Discontinuous measurement

Measurement that doesn’t capture every instance of behavior

Is used when continuous rate of measurement would be too time consuming or difficult in a certain setting (ex: therapy)

Partial Interval

Most common

Involves recording the presence or absence of a behavior during brief intervals of time. Useful for capturing behaviors that occur intermittently/ within varying frequencies Though it tends to overestimate the occurrence of behaviors as it only requires the behavior to occur once within the interval to be counted.

Ex: 10 minute is broken down into 20 or 30 second intervals. If behavior occurs at any time during the interval a “+” is written. If behavior doesn’t occur during interval, a “-” is written. The percentage of intervals with a “+” is calculated.

At every interval you mark + or -

Whole Interval Recording

Recording the presence/absence of a behavior if it occurs during the entire interval. Tends to underestimate the occurrence of behavior.

It becomes a “-” if it stops at any point

Ex: If the behavior occurs for 5 seconds but then stops it would be a “-”. However, if the behavior occurs throughout the whole 30 second interval it be a “+”

Momentary Time Sampling

Sampling a data collection procedure where the presence/absence of a behavior is recorded at the end of an interval, and then not recording for the rest of the interval. Provides the least amount of info

Does not require continuous monitoring of behavior throughout the interval, observations are made at predetermined points in time.

Ex: 10 minutes is broken down into 20-30 second interval, if behavior occurs during last 3 seconds of interval, it’s marked with “+”, if it doesn’t, “-”.

Permanent Product

Involves measuring behavior by examining the tangible outcomes or products resulting from that behavior.

Can be physical artifacts, completed tasks, or any other observable result that remains after the behavior has occurred.

Ex: Recording how many math problem a learner does correctly by counting the problems done correctly off a worksheet

Average/Mean

Measure of central tendency that represents the average of all set values

Add up all values and then divide the sum by the number in the dataset

Ex: 3, 4, 5, 4 (added up equals 16)

Mean: 16/4=4

Percentage

Represents a part of the whole in terms of hundredths

Ex: A student was observed to be on-task for 45 out of 60 intervals, the percentage would be 75% (45/60=0.75 × 100= 75%).

Artifact (data display)

Distortion/misrepresentation in the data, leading to potentially invalid measurement

Indirect Measurement

Involves using a substitute/proxy for directly observing the behavior

Observer Reactivity

Measurement error from when learner/participant is aware that they are being watched/evaluated

Observer Expectations

Expectations an observer holds about behavior being measured that can inadvertently influence how data is recorded

Level

Refers to the average or typical value of the data points across the graph

Might represent the baseline performance or behavior of a subject before any intervention or treatment (Phase A), as well as the level during and after the intervention (Phase B).

Trend

Refers to the general direction or pattern exhibited by the data points over time.

In an ABA line graph, the trend might indicate whether there is an increasing, decreasing, or stable pattern in the behavior or performance being measured.

Ex: if the graph shows a consistently increasing trend during the intervention phase, it suggests that the intervention is having a positive effect on the behavior or performance

Variability

Refers to the extent to which data points deviate from the average or typical value.

In an ABA line graph, variability might indicate fluctuations or inconsistency in the behavior or performance being measured.

High variability could suggest that external factors or conditions are influencing the behavior or performance unpredictably.