COSC 4315: Final Exam Review

RAG (Retrieval Augmented Generation)

a technique that enhances the accuracy and the reliability of AI models by retrieving and including information from specific and relevant data sources when generating responses

What does RAG allow LLMs to do?

RAG allows LLMs to access and incorporate external data sources like internal organizational data, journals or specialized datasets when generating responses

What are the benefits of RAG?

.Improved Accuracy: RAG reduces the risk of generating incorrect or irrelevant information by grounding LLMs in factual information from external information

.Enhanced Relevance: RAG ensures that generated content is tailored to specific contexts and user needs (basically it would be on topic of what is being asked)

.Access to Current Information: RAG allows AI models to access the latest information, despite it not being part of their original training data

.Cost Effective: RAG can be a cost effective way to improve LLM output without needing to waste money on extensive retraining

RAG vs LLMs

LLMs: They are powerful at generating text , yet they rely on their training data which can be limited or outdated

RAG: It allows LLMs to access external knowledge bases, making the generated content more accurate and relevant (it addresses the issues from LLM)

Why we prefer RAG?

RAG helps fix known challenges of LLMs:

.Presenting out of date or generic information when the user expects a specific, current response

.Creating a response from non-authoritative sources

.Creating inaccurate responses due to terminology confusion, wherein different training sources use the same terminology to talk about different things

Prompt Engineering

the practice of carefully crafting prompts for AI models to guide them towards producing desired outputs

Steps Involved in Prompt Engineering

1.Defining the task

2.Indentfying the inputs

3. Creating detailed prompts

4. Refining prompts

Creating Effective Prompts

clarity: prompts should be clear, concise, and easy to understand, avoiding ambiguity

context: providing adequate context is crucial for guiding the GPT model towards the desired outcome (this includes: background information, relevant examples, and specific details)

examples: including examples of the desired output can help the GPT model better understand expectations and generate more relevant responses

keywords: using keywords can help steer the GPT towards specific concepts or topics

tone: the tone of the prompt can influence the style and formality of the GPT model’s response

Roles

System role: set of instructions that define its general behavior and how it should interact with the user

.it sets the overall context, tone and any ethical guidelines or constraints for the AIs responses

ex: You are an helpful assistant…

User Role: represents the specific instructions or questions that the user provides to the AI system

ex: What is the weather today?

Prompt Engineering Techniques

Few shot prompting: language model is trained with a small number of examples, allowing it to generalize and perform tasks across various domains

Chain-of-thought: involves guiding a language model through a series of interconnected prompts, encouraging it to build upon previous responses

Prompt chaining: connects multiple prompts in a sequential manner

Tree of thought: structuring prompts in a hierarchal tree-like format, making the model to navigate through different branches of thought

Multimedia

Refers to several media types used together

Examples of Multimedia data

images

audio

video

graphics

animation

text

LiDar

voxels

Challenges of Multimedia Data Retrieval

.multimedia objects are, in general, large in size

.continuous multimedia objects have a temporal dimensional

.the meaning of multimedia objects is often unclear and at least subjected

.multimedia objects lacks obvious semantic structure

.difficult to identify components within media that could be used for retrieval or transaction processing

.to capture the meaning of multimedia objects, metadata must be derived

Metadata

data that describes multimedia objects

instead of searching for a multimedia object directly, search for the metadata that have been added to it

More metadata info

.desirable properties of metadata

.a description of multimedia object should be as complete as possible

.storage of metadata must not take too much overhead

. comparison of two metadata value has to be fast

Web Crawling

the process by which pages are gathered from the web (to index them and support a search engine)

Web Crawling Architecture

components of a web crawler architecture:

.the url frontier, containing urls yet to be fetched in the current crawl

.a dns resolution module that determines the web server from which to obtain a url to be fetched

.a fetch module that retrieves the web page at a url

.a parsing module that extracts the set of links from fetched web page

.a duplicate eliminations module that determines whether an extracted link is already in the url frontier or has recently been fetched

Duplicates

.The web contains multiple copies of the same content

.to keep down storage and processing overheads - search engines try to avoid indexing multiple copies of the same content

.one way to detect duplicates is through fingerprints

Fingerprint

a succinct (64-bit) representation of the characters on the page

.if two web pages have the same fingerprint, we test whether the pages themselves are equal

.its better/cheaper to compare two fingerprints than two webpages

Near duplicates

when the contents of one webpage are identical to those of another except for a few characters

Shingling

.define the k-shingles of a document d to be set of all consecutive of k terms

.intuitively, two documents are near duplicates if the sets of shingles generated from them are nearly the same

Cloud Computing

the practice of delivering on demand computing resources like hardware, storage and software over the internet

Infrastructure as a service (IaaS)

provides users with access to virtualized computing resources, such as servers, storage, and networking over the internet

(users have the most control over their infrastructure)

Platform as a Service (PaaS)

offers a platform for developing, running and managing applications without the complexity of managing the underlying infrastructure

(users focus on developing their applications, providers manages infrastructure, operating system and runtime environment

Software as a Service (SaaS)

delivers software applications over the internet, on a subscription basis

(users have the least control over the infrastructure but also the least responsibility)

Vertical scaling

allocate additional resources to VMs, requires a reboot, no need for distributed app logic, singlepoint of OS failure

Horizontal Scaling

application needs logic to work in distributed fashion

Abstraction

Virtualization separates the underlying hardware from the applications and operating systems that run on top of it, making it easier to manage and scale resources.

Virtual Machine

A virtual machine is a software-defined computer that runs on a physical computer with a separate operating system and computing resources

.The physical computer is called the host machine and virtual machines are guest machines

.Multiple virtual machines can run on a single physical machine. Virtual machines are abstracted from the computer hardware by a hypervisor