Relation Extraction and Sentiment Analysis

Rule-based Relation Extraction

If the mention of a hyponym and a hyperonym in a sentence are connected in a clearly distinguishable pattern, we can construct a rule to extract the relation.

Hand-written Rules Pros & Cons

+ Manually curated rules tend to have a high precision

+ Can be tailored to specific domains

-Manually curated rules often have low recall

-A lot of manual work is required to maintain rulesets

Bootstrap sampling

simulate having multiple different data sets (although you only have one) by repeatedly sub-sampling a single data set, for example to derive confidence intervals

Knowledge Base

In a knowledge base, relations are typically Encoded as triples

We can use such templates to identify patterns in which relations may occur in a corpus

Entity types can help us create pattern templates.

Distantly Supervised Relation Extraction

Starting with a few seed entities, find cooccurrences of these entities to identify patterns.

Use the patterns as templates for discovering new entities. Repeat.

Distantly Supervised Relation Extraction Pros and Cons

+ minimal manual input (a few seed entities or patterns) → semi-supervised approach (also called distant supervision)

+ System learns to extract new relations on its own

+ Improved recall

- A trade-off between precision and recall becomes necessary

- Recall is still not perfect - not all relations (always) occur in obvious patterns

Supervised Relation Extraction

Setup and design:

● Defining a set of relations for extraction

● Selecting a set of relevant named entities

Data:

● Compiling a representative training corpus

● Labeling named entities in the corpus

● Annotating relations between these entities (manually or pattern-based)

Training a classifier:

● Naïve Bayes

● SVM

Neural Language Models for Relation Extraction

Pre-trained contextual language models can be used to build relation extraction pipelines by leveraging what the model has learned about word relations (e.g. via attention).

● This is transfer learning. The model is

○ trained on an unsupervised task, and

○ fine-tuned for relation extraction.

● Adaptation of an annotation task:

○ Given a sentence with masked entities, the model is trained to label tokens with relation types.

● Adaptation of extractive question answering:

○ Given a sentence and the question the model outputs begin and end token indices of the relation in the sentence

Supervised Relation Extraction Pros and Cons

+high accuracy (both precision and recall) as long as we have sufficient labeled data

- Data hungry: requires a lot of labeled training data

- Often poor performance when adapting from one domain to another

Sentiment Analysis

Broadly speaking, sentiment analysis describes the tasks of identifying, extracting, quantifying, and studying affective states of the authors and expressed subjective information in text

Sentiment Analysis Applications

● Movies

● Products

● Public sentiment

● Politics

● Prediction

● Feedback:

Related detection / classification tasks for Sentiment Analysis

● Subjectivity

● Bias

● Stance

● Hate-speech

● Sarcasm

● Deception and betrayal

● Online trolling

● Polarization

● Politeness

● Linguistic alignment

Scherer Typology of Affective States

• Emotion: brief organically synchronized [...] evaluation of a major event

• Mood: diffuse non-caused low-intensity long-duration change in subjective feeling

• Interpersonal stances: affective stance toward another person in a specific interaction

• Attitudes: enduring, affectively colored beliefs, dispositions towards objects or persons

• Personality traits: stable personality dispositions and typical behavior tendencies

Sentiment Analysis: Method Overview

● holder of the attitude

● target of the attitude

● type of attitude: From a set of types, More commonly: simple weighted polarity

● Text containing the attitude

Challenges in the extraction of features

● Tokenization

● Stemming

● Negation

● Subtleties

Tokenization

Punctuation is typically removed or collapsed in preprocessing. But for sentiment analysis, it may contain valuable signals.

● Repetition of punctuation for emphasis

● Masking of expletives

● Emoticons may carry more sentiment signal than most words

Stemming

Stemmers heuristically identify word suffixes and strip them, with some regularization of the endings. This runs the risk of merging tokens with positive or negative connotation.

Stemming can be helpful for pooling the signal

Negation

Negation reverses the polarity of certain words

Workaround: Simple negation marking: append a _NEG suffix to every word that occurs between a negation and the next punctuation mark at the level of the current clause.

Drawback: The vocabulary size just doubled.

Subtlety

sarcasm,

thwarted expectations

subtlety

Sentiment Lexica

Some words or phrases can be clearly identified as carriers of a strong sentiment signal: positive, negative, neutral.

They are typically compiled

● manually in a dictionary, or

● automatically from a corpus.

Sentiment Analysis with Neural Language Models

pre-trained contextual language models can be fine-tuned for sentiment analysis:

● This is transfer learning. The model is

○ trained on an unsupervised task, and

○ fine-tuned for sentiment analysis.

● Adaptation of a classification task:

○ Given a sentence determine to which sentiment class it belongs.

● Adaptation of a regression task:

○ Given a sentence output a sentiment score