Analytic Techniques Review

Key Assumptions Check

• Usefulness: Particularly useful at the beginning of an analytic project.
• Purpose/Benefits:
  • To explain the logic of the analytic argument and expose any faulty logic.
  • To understand the key factors that shape a particular issue.
  • To stimulate deeper thinking and exploration about an issue.
  • To uncover hidden relationships and links between various key factors.
  • To identify developments or events that would necessitate abandoning a previously held assumption.
  • To prepare analysts for changed circumstances that could potentially surprise them.

Quality of Information Check

• Process: This is an ongoing and continuous process throughout the analytic project.
• Purpose/Benefits:
  • Can assist in detecting possible deception and denial strategies employed by an adversary.
  • Can identify crucial intelligence gaps and new requirements for intelligence collectors.
  • Can help policymakers understand the level of confidence analysts place on their analytic judgments.

Indicators or Signposts of Change

• Usefulness: Employed whenever an analyst needs to track an event over time to monitor and evaluate changes.
• Steps:
  • Identify a set of competing hypotheses or scenarios regarding the event.
  • Create separate lists of potential activities, statements, or events that would be expected to occur for each hypothesis or scenario.
  • Regularly review and update the indicators list to observe which indicators are changing.
  • Identify the most likely or most correct hypotheses or scenarios based on the number of observed changed indicators.

Analysis of Competing Hypotheses (ACH)

• Usefulness: Used when there are large amounts of data to absorb and evaluate. It is especially appropriate for controversial issues where analysts want to develop a clear record of the theories they have considered and how they arrived at their judgments.
• Purpose/Benefits:
  • Ensures that all information and argumentation are evaluated and given equal treatment or weight when considering each hypothesis.
  • Prevents the analyst from premature closure on a particular explanation or hypothesis.
  • Protects the analyst against innate tendencies to ignore or discount information that does not comfortably align with their currently preferred explanation.

Devil's Advocacy

• Usefulness: Used to challenge an analytic consensus or a key assumption regarding a critically important intelligence question.
• Steps:
  • Outline the main judgment and key assumptions, and characterize the evidence supporting the current analytic view.
  • Select one or more assumptions (whether explicitly stated or not) that appear most susceptible to challenge.
  • Review the information used to determine its questionable validity, whether deception is possibly indicated, or if major gaps exist.

Team A / Team B

• Usefulness: Applied to longstanding policy issues, critical decisions with far-reaching implications, or disputes within the analytic community that have obstructed effective cross-agency cooperation.
• Steps:
  • Set aside dedicated time for an oral presentation of the alternative team findings.
  • Convene an independent jury of peers to listen to the oral presentations and prepare to question the teams regarding their assumptions, evidence, or logic.
  • Allow each team to present their case, challenge the other team's arguments, and rebut the opponent's critique of its own case.

High Impact/Low Probability Analysis

• Usefulness: Employed when analysts and policymakers are convinced that an event is unlikely but have not given much thought to the potential consequences of its occurrence.
• Steps:
  • Define the high-impact outcome clearly and precisely.
  • Devise one or more plausible explanations or pathways that could lead to the low-probability outcome.
  • Insert possible triggers or changes in momentum, if appropriate. Examples include natural disasters, sudden health problems of key leaders, or new political shocks that might have occurred historically or in other parts of the world.
  • Brainstorm with analysts possessing a broad set of experiences to aid in the development of plausible but unpredictable triggers for sudden change.

What If Analysis

• Usefulness: Used when a judgment relies on limited information or unproven assumptions. This contrarian technique assumes an event has already occurred (with potential negative or positive impact) and then explains how it might have come about.
• Steps:
  • Develop a chain of argumentation based as much on logic as on available evidence to explain how the outcome could have arisen.
  • Specify what must actually occur at each stage of the scenario.
  • Consider the scope of both positive and negative consequences of each scenario and their relative impacts.

Brainstorming

• Usefulness: Primarily used at the beginning of a project to help generate a range of hypotheses about an issue.
• Guidelines:
  • Never censor an analyst's ideas, no matter how unconventional they might appear.
  • Find out what prompted the thought or idea.
  • Give sufficient time to conduct brainstorming correctly and effectively.

Outside In Thinking

• Usefulness: Applied at the conceptualization phase of an analytic project when the goal is to identify all critical, external factors that could influence how a particular situation will develop.
• Steps:
  • List all key external forces (technological, economic, environmental, political) that could have an impact on the topic.
  • Focus next on key factors over which an actor or policymaker can exert some influence.
  • Assess how each of these external forces could affect the analytic problem.
  • Determine whether these forces actually do have an impact on the particular issue based on the available evidence.

Red Team Analysis

• Usefulness: Particularly useful when trying to replicate the mindset of authoritarian leaders, terrorist cells, or other non-Western groups that operate under vastly different codes of behavior or motivations.
• Steps:
  • Analysts put themselves in the adversary's circumstances and react to foreign stimuli as the target would.
  • Develop a set of "first-person" questions that the adversary would ask.
  • Draft a set of policy papers in which the leader or group makes specific decisions, proposes recommendations, or lays out courses of actions.

Alternative Futures Analysis

• Usefulness: Beneficial when a situation is viewed as too complex or the potential outcomes are too uncertain to solely trust a single outcome assessment.
• Purpose/Benefits:
  • Provides an effective means of weighing multiple unknown or unknowable factors and then presenting a set of plausible outcomes.
  • Can help bound a problem by identifying plausible combinations of uncertain factors.
  • Provides a broader analytic framework for calculating the costs, risks, and opportunities presented to policymakers by different potential outcomes.