Hallucinations in LLMs and Browser Extension Supply Chain Security

The Evolving Attack Surface of Modern Browsers

Palo Alto Networks

• The Browser as an Operating System: In the current technological landscape, the web browser has evolved into a de facto operating system. This transformation makes understanding its attack surface critical for security.

• Research Focus: Recent security research emphasizes the intersection of Large Language Model (LLM) hallucinations and browser extensions. The goal is to identify vulnerabilities and "ghost" extensions before malicious actors can exploit them.

• Browser Extension Risks: Browser extensions are inherently dangerous due to several factors:

  • Excessive Permissions: They often operate with broad access rights.

  • Data Access: Extensions can see sensitive information, including session tokens and cookies.

  • Front Row to the OS: A compromise in a browser extension provides attackers with direct, high-level access to the user’s broader operating system.

  • Simple Exploitation: Attackers do not necessarily need to breach a system directly; they can simply trick an AI via hallucinations to gain entry through the "front door."

Understanding LLM Hallucinations

• Definition and Prevalence: A hallucination occurs when a chatbot or LLM provides a false fact with high confidence. This is a persistent issue that most users of AI technologies have encountered.

• Limitations of Current Research: Most existing benchmarks and research focus on hallucinations in the context of summarization. There is a lack of focus on hallucinations regarding supply chain security and related software artifacts.

• Conceptual Example: The Martian Sunset:

  • When asked about the color of the sunset on Mars, LLMs may generalize Earth-based facts and knowledge of the Martian landscape to incorrectly state the sunset is red.

  • Physical Reality: The actual color of a sunset on Mars is blue.

  • Scientific Explanation: Blue light has a shorter wavelength, while red light has a longer wavelength. On Mars, the Martian sky and atmospheric particles interact with these wavelengths differently than on Earth. Photographic evidence from Martian rovers confirms the blue hue of the sunset.

  • Physics Representation: Light properties can be characterized by their wavelength $\lambda$. In this context, blue light $\lambda_{blue}$ is shorter than red light $\lambda_{red}$, affecting how scattering occurs in the Martian atmosphere.

Hallucination Fault Zones in Supply Chain Security

• The Proliferation of Extensions: The advent of AI has led to a massive spike in the creation of browser extensions. Current data shows approximately $20,000$ new extensions created every month, a volume that previously took an entire year to accumulate.

• Recency Detection Failures: LLMs struggle with recency. They cannot accurately distinguish between legacy data and current software releases, often hallucinating recommendations for tools that may no longer be secure or relevant.

• Snapshot Learning Limitations: LLMs are not "snapshot learners." They lack a real-time understanding of when specific extensions are deleted from stores. Consequently, they continue to recommend extensions that no longer exist.

• Branding Issues: Many extensions lack established brands. This absence of a recognizable identity leads LLMs to confidently hallucinate names and functionalities for extensions that do not exist.

• Historical Examples: LLMs have been observed hallucinating extensions that were previously deleted due to being marked as malware in official stores.

Security Implications and Real-World Exploitation

• Hallucination Scenarios:

  • Unclaimed Extensions: LLMs may recommend extensions that have never existed or are currently "unclaimed." Attackers can identify these hallucinated names and register them to distribute malicious code.

  • PyPI and Package Hallucination: Research has demonstrated that these issues are not limited to browsers; they also appear in repository systems like PyPI (Python Package Index) and in domain name suggestions.

• Malicious Activity Trends:

  • Ongoing monitoring of highly hallucinated extensions has revealed that about a dozen such names were eventually registered by external parties and used for malicious activities.

  • Rescoring and monitoring these specific vulnerabilities is necessary to track how attackers weaponize AI misinformation.

• Critical Recommendations:

  • Users and security professionals must never blindly trust the recommendations provided by LLMs or chatbots regarding software installation.

  • Fact-checking is an inherent requirement when dealing with LLMs, as hallucinations are a fundamental characteristic of the technology that may never be fully eliminated.

  • Double-checking specific software artifacts and extension names is the only reliable defense against AI-driven supply chain deception.