PSYU/X3330 Week 11: Comprehensive Notes on Signal Detection Theory

Overview of Signal Detection Theory (SDT)

• Definition: Signal Detection Theory is a framework used to characterize performance when decisions must be made in the presence of uncertainty. This uncertainty arises in tasks involving discrimination, identification, and classification.

• Origin: It was originally formulated for engineering applications, such as detecting a genuine radar signal amidst random electronic fluctuations.

• Psychological Application: In psychology, it was applied to overcome the limitations of Classical Threshold Theory (CTT). It is frequently used for studying sensory systems, as it allows researchers to separate an observer's sensitivity from their decision strategy (bias).

The Lie Detection Task (Tutorial Component)

• Methodology: Students participate in a task where they generate two sentences about themselves that appear surprising. These can be:

  • Both true.

  • Both lies.

  • One of each (one truth, one lie).

• Marking the Ground Truth: Students must record which statements are true and which are lies (marked "A" and "B") and provide this information to the lecturer.

• The Decision Process: Listeners hear the statements and rate their confidence regarding whether each statement is a lie using a 6-point Likert scale:

  1. Definitely true

  2. Probably true

  3. Possibly true

  4. Possibly a lie

  5. Probably a lie

  6. Definitely a lie

• Binary Conversion: To analyze the data using standard SDT metrics, the 1–6 Likert responses are converted into a dichotomous (binary) response:

  • Responses 1–3 = "Truth"

  • Responses 4–6 = "Lie"

Classical Threshold Theory (CTT) and its Limitations

• Core Concept: CTT is based on a fixed "threshold." If stimulus intensity is greater than the threshold ($I > T$), sensation occurs. If it is less ($I < T$), there is no sensation.

• The Psychometric Function:

  • Ideally, CTT predicts a "step function."

  • In reality, experimental factors like random noise exist. External noise includes variations in stimulus intensity, while internal noise refers to random changes in the observer's sensitivity.

  • Due to noise, the step function becomes a smooth, sigmoidal psychometric function.

• Threshold Specification: The detection threshold is typically specified as the value where the stimulus is detected $50\%$ of the time.

  • Example: A light with brightness of $4\,cd/m^2$ or a sound at $4\,dB\,SPL$

• Critical Problems with CTT:

  • Threshold estimates are affected by non-sensory variables like criterion/response bias.

  • Prevalence Effect: If a stimulus is presented more frequently, observers are more likely to report detecting it, shifting the location of the psychometric function and resulting in a different (inaccurate) threshold measurement.

SDT Data: The 2x2 Decision Matrix

• In any SDT task, there are two types of stimulus conditions (Signal Present vs. Noise Alone) and two types of responses (Yes/Signal vs. No/Noise).

• The Categories of Outcomes:

  • Hit (H): Correctly identifying a signal when it is present (e.g., saying "that's a lie" when the sender told a lie).

  • Miss (M): Failing to detect a signal when it is present (e.g., saying "that's a truth" when the sender told a lie).

  • False Alarm (FA): Reporting a signal when only noise is present (e.g., saying "that's a lie" when the sender told the truth).

  • Correct Rejection (CR): Correcting identifying that no signal is present (e.g., saying "that's a truth" when the sender told the truth).

• Matrix Representation:

• Data Transformation:

  • Total counts are converted into proportional response rates.

  • The Sum of proportions for SN trials ($Hit + Miss$) must equal $1$.

  • The Sum of proportions for N trials ($False\,Alarm + Correct\,Rejection$) must equal $1$.

Graphical Representation of Signal and Noise

• Signal: Any stimulus (light, sound, medical scan tumor) that causes an increase in internal response (e.g., neural activity).

• Noise: Always present, consisting of random variations. It includes internal noise (sensory system) and external noise (stimulus-related factors).

• Probability Distributions: SDT represents these as two distributions on a graph where the x-axis is "Internal Response (arbitrary units)" and the y-axis is "Probability (%)":

  • N Distribution: Describes random variation of internal responses to noise alone.

  • SN Distribution: Describes the internal response to Signal + Noise.

  • On average, the internal response is larger for SN than for N.

Sensitivity ($d'$) and Criterion ($c$)

• Decision Process: The receiver sets an internal criterion. If the internal response exceeds this criterion, the observer says "Yes."

• Visualizing Outcomes on Graph:

  • Hit Rate: Area of the SN distribution to the right of the criterion.

  • Miss Rate: Area of the SN distribution to the left of the criterion.

  • False Alarm Rate: Area of the N distribution to the right of the criterion.

  • Correct Rejection Rate: Area of the N distribution to the left of the criterion.

Calculating Sensitivity ($d'$)

• Sensitivity is determined by the overlap between N and SN distributions. It is affected by the difference between means (separation) and the standard deviation (spread).

• Formula (General): $d' = \frac{\text{separation}}{\text{spread}}$

• Formula (Standardized):

  • $d' = \frac{M_{SN} - M_N}{\sigma(N)}$

  • $d' = z(H) - z(F)$

  • Where $z()$ refers to the z-score (using NORM.S.INV() in Excel).

• Significance: $d'$ is invariant when bias changes, making it a purer measure of sensitivity than CTT thresholds.

Calculating Criterion and Bias ($c$)

• Bias represents the tendency to favor one response over the other regardless of the stimulus.

• Formula: $c = -0.5 \times [z(H) + z(F)]$

• Criterion Manipulation:

  • Reward for Hits: Encourages a Liberal Criterion (lowering the threshold for saying "yes"). Increases Hits but also increases False Alarms.

  • Punishment for False Alarms: Encourages a Conservative Criterion (raising the threshold for saying "yes"). Decreases False Alarms but also increases Misses.

Receiver Operating Characteristic (ROC) Curves

• Definition: A plot of the Hit Rate ($y$-axis) versus the False Alarm Rate ($x$-axis) across different criterion levels for a fixed sensitivity ($d'$).

• Key Features:

  • Chance Line: A diagonal line where $HR = FAR$. This indicates the observer cannot distinguish signal from noise ($d' = 0$).

  • Curve Position: As $d'$ increases, the ROC curve shifts toward the upper-left corner.

  • Asymmetry: If the ROC curve is not symmetrical, it implies the standard deviations of the N and SN distributions are not equal.

  • Bypassing the Chance Line: If performance is below the chance line ($FAR > HR$), the subject may be misunderstanding the task.

Alternative SDT Procedures

SDT can be adapted to various tasks beyond the simple "Yes-No" design:

• 2-Alternative Forced Choice (2AFC): Two alternatives occur; the observer must report their order or which interval contained the signal.

• Same-Different: Two stimuli are presented (S1, S2) in four possible pairs (S1, S1 $S2,S2$ , <S1, S2$<S2,S1>$ he observer identifies them as "same" or "different."

• ABX (Matching to Sample): The observer decides whether A or B matches the sample X.

• Oddity: Three stimuli are shown (A, B, C); the observer identifies the one that is different.

• Classification: Category judgment, which can be one-dimensional or multi-dimensional.

Why Measure Thresholds if SDT Exists?

• Intuition: Thresholds are concrete (e.g., "I can see 2% contrast") whereas $d'$ is abstract (e.g., "0.12 sensitivity units").

• Hybrid Designs: Researchers use "hybrid" designs like 2AFC for threshold measurement to address bias concerns raised by SDT while maintaining the intuitive nature of thresholds.

Test Yourself Scenarios

The Wink Scenario

• Context: You are at a bar and think someone winked at you. You must decide whether to approach or not.

• Conservative Criterion: "That gorgeous creature can’t possibly have been winking at me. No-one ever does… I’m going home." (Requiring very high evidence before saying "yes").

• Liberal Criterion: "Even if I have only the slightest suspicion that they winked at me, I’m going to allow them the pleasure of meeting me."

Prank Calls Scenario

• Context: Emergency services receive many prank calls. Ignoring a real emergency is more costly than responding to a fake one.

• Application: Emergency services deliberately set a very liberal criterion. They would rather have a high False Alarm rate (responding to a prank) than increase the Miss rate (ignoring a genuine emergency).

Reference Materials

• Stanislaw, H., & Todorov, N. (1999): "Calculation of signal detection theory measures." Behav Res Methods Instrum Comput, 31(1), 137-149.

• George Gescheider: Psychophysics: The Fundamentals.

• Kingdom, F. A. A., & Prins, N. (2009): Psychophysics: A Practical Introduction.

• Macmillan & Creelman: Detection Theory, A User’s Guide.