Challenges in NLP

What is NLP?

  • NLP = Natural Language Processing.

  • Field focused on building systems that process, understand, or generate human language.

  • An application is considered NLP when it requires knowledge about human languages (syntax, semantics, pragmatics, etc.).

Examples of NLP Applications

Information Retrieval

  • Searching for relevant documents in large collections.

Document Classification

  • Assigning documents to categories (e.g., spam vs. non‑spam, topic classification).

Question Answering

  • Producing short, direct answers to user questions.

Text Summarisation

  • Creating concise summaries of one or multiple documents.

Sentiment Analysis

  • Detecting opinions, emotions, or attitudes.

  • Used in product reviews, social media, hate speech detection.

Machine Translation

  • Translating text between languages.

  • One of the earliest motivations for NLP research.

Natural Language Generation

  • Producing coherent, meaningful text (e.g., chatbots, report generation).

Conversational Agents

  • Dialogue systems.

  • Speech recognition.

  • Text‑to‑speech and speech‑to‑text.

Authoring & Marking Tools

  • Spell checking.

  • Grammar/style checking.

  • Automated essay marking.

Why NLP is Challenging

Text is Unstructured

  • Unlike structured tables (e.g., spreadsheets), natural language is free‑form.

  • Documents vary in:

    • Style

    • Length

    • Grammar

    • Formatting

    • Vocabulary

  • Machines must infer structure that humans understand intuitively.

Variability

Many ways to express the same meaning.

Example (same meaning, different phrasing):

  • “X succeeds Y as chairperson.”

  • “Y is replaced by X as chairperson.”

  • “Company Z appoints X after Y steps down.”

This variability makes NLP tasks like classification, summarisation, and translation difficult.

Ambiguity

Language is inherently ambiguous.

Types of Ambiguity

Lexical Ambiguity

A single word can have multiple meanings or grammatical categories.

Examples:

  • duck → noun (bird) or verb (lower head)

  • bank → financial institution or river bank

  • apple → fruit or company

  • Acronyms vary by context (e.g., “NLP” = natural language processing OR neuro‑linguistic programming)

Lexical‑Semantic Ambiguity

Words have multiple senses.

Example:

  • make → create, cook, cause

Sentence-Level Ambiguity

A sentence can be interpreted in multiple ways.

Example: “I made her duck.”

Possible interpretations include:

  • Cooking duck for her.

  • Cooking her duck (the one she owns).

  • Creating a duck (e.g., a model) for her.

  • Causing her to duck (lower her head).

  • Turning her into a duck (magical transformation).

Ambiguity arises from:

  • Word senses

  • Parts of speech

  • Syntactic structure

  • Pronoun roles (possessive vs. dative)

Syntactic Ambiguity

Ambiguity caused by different possible grammatical structures.

Attachment Ambiguity

  • “I saw the girl with the telescope.”
    → Who has the telescope?

Coordination Ambiguity

  • “Old men and women.”
    → Are women old too?

  • “Mother and baby in pram hit by car.”
    → Who was in the pram?

Local Ambiguity

Temporary ambiguity until later words clarify meaning:

  • “The cat that watched the dog ate.”

  • “Police help dog bite victim.”

  • “The old man the boat.” (garden‑path sentence)

Basic Linguistic Concepts

Parts of Speech (POS)

  • Open class: nouns, verbs, adjectives, adverbs

  • Closed class: pronouns, prepositions, determiners, auxiliaries

Linguistic Features

  • Person

  • Number

  • Tense

  • Aspect

  • Voice

These features help determine meaning and grammatical structure.

Lexical Semantics

Study of word meaning and relationships.

Key concepts:

  • Synonymy: similar meaning

  • Antonymy: opposite meaning

  • Polysemy: multiple related meanings

  • Homonymy: multiple unrelated meanings

  • Hyponymy: hierarchical relationships (e.g., “sparrow” is a hyponym of “bird”)

Syntactic Ambiguity (Detailed)

Verb Argument Structure

The verb make can take different grammatical patterns:

Transitive

  • Verb + direct object
    → “I cooked waterfowl belonging to her.”

Ditransitive

  • Verb + two objects
    → “I made her into a duck.”

Action‑Transitive

  • Verb + object + verb
    → “I caused her to duck.”

Additional Challenges in NLP

  • Idioms and fixed expressions

  • Proverbs

  • Irony and sarcasm

  • Context and world knowledge

  • Spoken language (noise, accents, disfluencies)

  • Non‑verbal communication cues

  • Large‑scale data

  • Multilingual text

  • Mixed scripts or noisy text (e.g., OCR errors)