Deterministic versus Probabilistic Systems and Large Language Models

Deterministic versus Probabilistic Systems

• Deterministic Processes:

  • A deterministic system or process produces the exact same output every single time when given the exact same input and starting conditions.
  • The internal process execution, the path taken to reach the output, and the final output itself are entirely known, fixed, and predictable.
  • Example: In computer coding, writing a basic Python script that uses fundamental functions to calculate and output the sum of two numbers, such as:     $2 + 2 = 4$     This script will consistently produce $4$ as its output every single time it is run without variation.

• Probabilistic Processes:

  • A probabilistic system incorporates elements of randomness, chance, statistical guessing, or variables that change over time.
  • Rather than following a fixed deterministic path, a probabilistic system operates on likelihoods—predicting what a process will "probably" do or how it "usually" behaves.
  • Because of the built-in variable of randomness or temporal change, the process execution and the final output are more complex and variable.

Large Language Models and System Architecture

• Probabilistic Nature of Large Language Models (LLMs):

  • Large language models belong to the category of probabilistic tools.
  • They do not generate the exact same output from the exact same input every time.

• Behavioral Demonstration of Non-Deterministic Variation:

  • An experiment demonstrating this architecture involves using an AI chatbot:
  1. Enter a specific prompt into the AI chatbot.
  2. Clear the browser cache and clear the conversation history.
  3. Start a completely new conversation session.
  4. Type the exact same prompt into the chatbot.
  • Result: The chatbot will yield a slightly different answer because the underlying model is probabilistic rather than deterministic.

• Functional Limitations and Strengths:

  • Limitations: Probabilistic models are inherently unsuited for tasks requiring identical, repetitive precision. For instance, calculating the addition of two numbers reliably requires a deterministic tool (like a Python script) to ensure the exact correct answer every time.
  • Strengths: A probabilistic tool possesses high flexibility and can accomplish a wide variety of different tasks, though it will execute them slightly differently upon each iteration.

Practical Implications and Applications

• Dual Impact of Output Variability:

  • The dynamic changing nature of probabilistic outputs can be an advantage in contexts requiring variety or creativity.
  • Conversely, variability can act as a harm when consistency and strict repeatability are required.

• Real-World Implementation:

  • Large language models are being increasingly integrated across digital platforms as automated customer service channels.

• Learning Environment Dynamics:

  • Being in an educational or learning environment means accepting the potential of being wrong while taking guesses and acquiring new knowledge.

Session Logistics and Dialogue

• Audience Interaction and Presentation Length:
  • Presentation procedures involve audience participation actions, such as navigating to websites for more information, raising hands, or speaking up.
  • The total planned speaking duration for the presentation block spans $70\,\text{minutes}$.