Reinforcement Learning, Social Media Algorithms, and Ethics in AI

A quiz is scheduled for next week, with a potential bonus quiz sometime before the exam.
Optimization was considered but its inclusion is uncertain.

Reinforcement learning was once considered the pinnacle of machine learning.
Augmented techniques with pre-trained models have become more effective with the abundance of data.
Reinforcement learning algorithms are simple but computationally expensive, requiring significant memory and processing power.
This expense translates to business costs, including storage, processing power, and electricity, especially with cloud computing's pay-per-use model.

Social media algorithms are often black boxes, making analysis speculative.
Personalized algorithms recommend content, with periodic updates and potential user movement between groups.
Algorithms can move users into hyper-specific groups based on interests.
New content affects user experience within the existing algorithm.
Algorithms may change the reach of posts based on user interaction.
Major algorithm updates occur periodically, impacting user experience significantly.
The learning rate in reinforcement learning influences how quickly algorithms adapt to new data.
Big algorithm changes are expensive and difficult, favoring incremental adjustments within the same framework.

RAG is a technique described as "baby reinforcement learning."
It involves adding a layer on top of a large language model for customized results.
Instead of retraining the entire model, RAG retrieves and incorporates new information.
A base model, like ChatGPT, is used by everyone, but paid versions offer a customized layer with user-specific data.
RAG acts as if the prompt includes all previous information, creating a semi-customized experience.
This approach is similar to ensemble models, where a customized model is retrained on the results of a larger model.
The aim is to customize the algorithm while keeping the main algorithm the same.
Example: Feeding BCIT website information or personal writing samples to ChatGPT to tailor responses.
Free versions of ChatGPT do not retain past interactions, whereas paid versions do.

Newer methods for model customization are emerging.
*Computational Augmented Generation (CAG) was mentioned as a newer method
The main challenge is optimizing storage, memory, and processing to reduce electricity consumption.
Many AI companies are not yet profitable due to high electricity costs.
The goal is to develop more efficient algorithms that can be supported by user fees.
Pricing models may shift to per-use rather than per-person.
The focus is on creating "good enough" AI models that are cost-effective.

AI development is outpacing legal frameworks, leading to lawsuits.
Canada is attempting to pass AI governance legislation for responsible use.
AI systems are being challenged in court to establish precedent.
Examples:
- Air Canada was sued for incorrect information provided by its chatbot.
- Lawyers faced contempt charges for using AI that fabricated legal precedents.
- The New York Times and authors like Stephen King have sued OpenAI.
Companies are forming AI councils with legal, technical, and executive members to promote responsible AI.
Human-in-the-loop AI ensures human oversight of AI decisions.
It's important to consider how to ensure the correctness of AI and what types of errors it may make, especially in applications like chatbots and medical testing.
- A major concern with chatbots is to ensure they don't make promises that the company can't keep (e.g., refund policies for Air Canada).

Continuous improvement is a way to think of reinforcement learning.
The Tic-Tac-Toe example illustrates the principles of game theory.
Game theory includes:
- Classical game theory (economics, games of chance).
- Combinatorial game theory (games of no chance like tic-tac-toe and Connect Four).
Winning Ways for your Mathematical Plays by John Conway and Richard Guy is a reference for learning about these games.