Generative AI and Large Language Models Vocabulary

Generative AI (GenAI): A branch of Artificial Intelligence designed to create content such as text, images, audio, video, code, and $3D$ models.
Large Language Models (LLMs): A type of machine learning model that performs natural language processing (NLP) tasks. These models learn context and understanding through neural networks called Transformers.
Parameters: Numerical values that define model behavior, adjusted during training to optimize coherent text generation. Current leading models use hundreds of billions of parameters (e.g., GPT-4 uses approximately $1.76$ trillion MoE).
Tokens: Basic units of text or code used for processing. A common rule of thumb is that $1$ token corresponds to approximately $4$ characters, or $100$ tokens $\approx$ $75$ words.

Training Method: Models utilize self-supervised learning to predict the next token in a sequence based on surrounding context.
Context Window: The limit of text a model can process in one instance.
- GPT-4: Supports up to $128,000$ tokens.
- Gemini 1.5: Supports up to $1$ million multimodal tokens, capable of reasoning across massive documents like the $402$ -page Apollo 11 transcripts.
Market Growth: The Large Language Model market size is projected to grow from approximately $10.57$ billion USD in $2025$ to $149.89$ billion USD by $2035$ .

Definition: The process of constructing inputs to language models to generate useful, specific outputs.
Learning Paradigms:
- Zero-shot: Prompting with instructions but no examples.
- One-shot: Including one demonstration of the task.
- Few-shot (in-context): Including multiple demonstrations ( $10+$ ) for better accuracy.
CIDI Framework: A structured approach to prompting: Context, Instructions, Details, and Input.
Adjustable Variables: Users can tweak "temperature" (creativity), maximum response length, and frequency/presence penalties to discourage repetition or encourage diversity.

Text and Reasoning: OpenAI (ChatGPT), Google (Gemini), Meta (Llama 2), Anthropic (Claude 2.1), and Mistral.
Visual and Video Generation:
- Images: Midjourney, Stable Diffusion, DALL-E 3.
- Video: Runway Gen-2, Sora (text-to-video), and EMO AI (audio-to-video).
Coding and Audio:
- Code: GitHub Copilot (utilizes OpenAI Codex).
- Audio/Voice: VALL-E, Resemble.ai, ElevenLabs, and Suno for music generation.
Gaming: Ludo.ai, FRVR Forge, Unity Muse, and Scenario-labs for style-consistent game assets.

Organizational Adoption: In Ireland (Feb 2024), $50\%$ of organizations use GenAI, with multinationals utilizing it $30\%$ more than indigenous organizations.
Educational Shift: AI is viewed as a defining technology comparable to the printing press, requiring a focus on human skills, creative thinking, and a "human in the loop" approach.
Systemic Change: Impact highlights include reducing "busy work" for educators, allowing more time for one-to-one instruction, and the need to rethink assessment to avoid bypassing useful cognition.
Ethical Constraints: Researchers like Emily M. Bender highlight the dangers of "stochastic parrots," questioning if language models can become too large and biased.