Hoofdstuk 5: IaC (copy)

IaC: what & why

• figuur

Context: operations

Initial deployment

• Provisioning servers (creating VM, installing OS, setting up network)

• Configuring OS (SSH, firewall, user accounts, services, monitoring)

• Installing services (webserver, database)

• Deploying software (CI/CD)

Maintenance

• Upgrading servers ([virtual] hardware, networks, firmware)

• Upgrading OS (Kernel, OpenSSL, Java runtime)

• Upgrading services (webserver, database)

• Upgrading software (CI/CD)

IaC: Automate Everything!

Manual approach (before automation)

• Issues

• Configuration drift

  • servers start out identical but changes accumulate over time

• Scaling

  • handling more complex infrastructure

Manual approach (before automation)

• What if stuff goes wrong: Fault tolerance vs Disaster recovery

Fault tolerance:

• handing known failure modes with

  • More instances

  • Backups

  • Robustness design patterns

  • Manual intervention to get duplication back

Disaster recovery

• handling complete disruptions with

  • Everything as code

  • Get back up asap

Automation attemp 1:

• Scripts

• Issues (figuur)

Automation attempt 2:

• Code describing infrastructure

• verschil met scripting (figuur)

History: CFEngine

Created in 1993 by Mark Burgess (Theoretical Physics)

• Convergence: user describes final state, software decides what actions are needed to go to that state.

  • Declarative description of desired state

  • Unpredictable initial state, predictable end state

Imperative vs Declarative

• Imperative (The "How"): You provide explicit, step-by-step commands or actions to achieve a goal. An example is writing a traditional script or a for loop where you tell the system exactly what to do at each step.

• Declarative (The "What"): You describe the desired end state you want to achieve, and the underlying system figures out the necessary steps to get there. For example, in Kubernetes or Infrastructure as Code, you declare your intent (e.g., "I want 3 instances of this application running"), and the system continuously works to reach and maintain that state without needing step-by-step instructions.

Linux configuration is imperative!

• Every config management tool wraps imperative

• Linux commands in declarative interfaces!

• But! efforts underway to make userspace completely declarative

Aside: declarative linux NixOS

Desired State Principle

• Is a declarative config management design pattern.

• User defines in a model what the desired state is. The system takes the actions in order to get the server in that state.

• Example: Kubernetes

• Unpredictable initial state → predictable end state

Idempotence and Convergence

Describes “How” the system achieves the desired state

• idempotence: Running an action twice gives the same result

• Convergence: Check if the action is needed.

  • faster “no-ops”

Summary: why IaC

• Improve consistency and reliability

  • Reduce human error

  • Reduce environment drift

  • Self-documenting infrastructure

• Scale sysadmins

• Speed up deployment, change and recovery

  • Increase reusability

  • Workflow automation

IaC landscape (overview)

Ansible

• Algemeen

• Hybrid: imperative execution of declared changes (YAML)

• Agentless.

  • Client uses SSH to copy code to remote machine and run it.

• You modify servers by running modules.

  • Each module is a Python script that runs on the servers.

  • Modules take input.

    • *Many modules use desired state principle.

• Cross-platform

  • Ubuntu most popular platform

Ansible

• model terminology

• Task: single execution of a module

• Playbook: File with a collection of tasks

  • Subdivided in plays

  • Statically reusable

  • imperative list of declarative changes

• Role: Templated collection of tasks and related content

  • Dynamically reusable

  • Can be configured to fit specific need

  • Shared online

Ansible

• Ecosystem

• Ansible Engine: Client CLI tool

• Ansible Galaxy: Online repository of modules and roles

  • Install Kubernetes, manage LDAP, deploy Minecraft server,..

  • Quality varies

  • Commercial/private version: Ansible Automation Hub

• Ansible Tower: Control server

  • Role Based Access Control

  • Job Schedulin

  • Dashboard and inventory management

Terraform

• algemeen

• Declarative (HCL or json)

• You change virtual infrastructure

• Agentless

• Compatible with most platforms and clouds

• Terraform is not open source

  • Business Source License (BSL)

  • Source-available license

  • Cannot use product if you compete with Hashicorp offering

  • All code Open Source after 2 years

Terraform

• configuration

• modules

A configuration contains multiple resources, a provider and Terraform config

• Resources can be “locked”

• Resources are cloud-specific

Terraform modules = reusable configurations

Ansible and terraform possible together

Terraform

• OpenTofu

Open Source fork of old Terraform code

• Compatible with Terraform 1.5.x

• Linux Foundation project

• Slower development but larger community

Chef

• Algemeen

Chef Infrastructure

• Hybrid: imperative execution of declared changes (Ruby)

• You modify servers using Resources

  • * Many Resources use desired state principle

• Agent (Chef Infra Client) connects to Chef Infra Server to retrieve models

• Cross-platform

Chef

• Cookbooks

Templated model of desired state

• A Cookbook contains multiple Recipes

• A Recipe contains multiple Resources

• A Resource describes the desired stat

Community code is shared on the Chef Supermarket

Puppet

• algemeen

• ecosystem

• Declarative (Puppet programming language)

  • Execution order not guaranteed

• Agent connects to master to fetch desired state

• You modify servers by defining Resources

• Cross-platform

Puppet Enterprise

• Scalability

• UI

• Role Based Access Control

• Support

Juju (Canonical)

• algemeen

• Declarative (YAML)

• Agents get the desired state from Controller

• You manage infrastructure by creating models and modify servers by deploying Charms

• Focus on complex, interdependent applications

• Compatible with multiple platforms

  • Ubuntu best supported

Juju

• relations

• Dynamic reconfiguration based on attached services

• Great for highly modular software

Juju

• charms

• High-level pieces of automation code downloaded from their webiste

• Any language (Python is best supported)

• Deploying applications, not changing config

• Using Juju often requires writing Charms

Possibility to reuse existing automation code

Desired State Configuration

• algemeen

• Declarative (PowerShell objects)

• You modify servers using Resources

• Agent (Local Configuration Manager)

• Compatible with multiple platforms

• PowerShell Gallery hosts a wealth of DSC resources

Kubernetes

• algemeen

• verschil met H4 (5 belangrijke)

“Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications.”

Hier zijn de belangrijkste nieuwe zaken die in hoofdstuk 5 over Kubernetes worden uitgelegd:

• Kubernetes IS een IaC-tool: De lesgever benadrukt dat je traditionele IaC-tools zoals Ansible of Terraform niet moet gebruiken om applicaties binnenin Kubernetes te beheren. Omdat Kubernetes fundamenteel gebouwd is op IaC-principes (zoals het declaratief beschrijven van de "desired state" in YAML), is het zelf al een uitstekende tool voor configuratiebeheer en CI/CD. Het gebruik van Ansible of Terraform in combinatie met Kubernetes is voornamelijk nuttig om het Kubernetes-cluster zélf te installeren of om applicaties te beheren die (nog) niet geschikt zijn voor Kubernetes.

• Beheer van externe cloud resources (In plaats van Terraform): Kubernetes wordt steeds vaker gebruikt om niet alleen containers, maar ook échte cloud resources te beheren, zoals virtual machines of Database-as-a-Service. In plaats van een gewenst model in Terraform te schrijven, kun je dit direct in Kubernetes definiëren. De Kubernetes operators en de cloud controller manager communiceren vervolgens met de API van de cloud provider (zoals AWS of Azure) om deze infrastructuur voor jou op te zetten.

• Kubernetes die Kubernetes uitrolt (Cluster API): Een specifiek nieuw concept is het gebruik van een Kubernetes-cluster om meer Kubernetes-clusters op te vragen en te beheren. Via de zogenaamde "Cluster API" kan Kubernetes communiceren met hypervisors (zoals VMware of LXD/MAAS) om virtuele machines of fysieke servers te voorzien en daar Kubernetes op te installeren. Dit is exact het mechanisme dat achter de schermen wordt gebruikt voor de xNative Lab testomgeving waar je in de labosessies mee zult werken.

• Een gestandaardiseerde API voor automatisatie: Kubernetes groeit uit tot een gestandaardiseerde API om te interageren met infrastructuur. CI/CD systemen gebruiken steeds vaker uitsluitend de Kubernetes API (via kubectl) om deployments aan te sturen. Omdat applicaties via deze API kunnen worden beheerd, ontstaat er een ecosysteem waarbij de ene Kubernetes-applicatie geautomatiseerd andere applicaties beheert.

• Aansturen van externe hardware: Waar operators in hoofdstuk 4 vooral interne K8s-zaken afhandelen, wordt in hoofdstuk 5 vermeld dat operators ook externe systemen kunnen configureren. Zo kan Kubernetes bijvoorbeeld communiceren met een externe Citrix ADC ingress controller of zelfs fysieke netwerkswitches configureren om virtuele netwerken (VLAN's) op te zetten.

Kubernetes

• as a platform

• as an API

IaC Design Patterns

• Unattended automation

Iac Design Patterns

• Re-use & promote definitions

IaC Design Patterns

• Cattle not Pets

Pets are indispensable or unique systems that can never be down. Typically manually built, managed and hand fed. During a failure event, pets need to be manually nursed back to health.

Cattle are arrays of more than two servers, that are built using automated tools, and are designed for failure, where no one, two, or even three servers are irreplaceable. Typically, during failure events no human intervention is required as the array exhibits attributes of “routing around failures” by restarting failed servers or replicating data through strategies like triple replication or erasure coding.

IaC Design Patterns

• Issue: duplication

• 3 opl

IaC Design Patterns

• Issue: Shared elements

Webassembly for Edge and IoT devices

WebAssembly is binary code

• Compiled for a “virtual” CPU

• Runs in a runtime: Browser, Node.JS, wasmtime

  • Just In Time (JIT) - compiled by runtime while running

  • Ahead Of Time (AOT) - compiled to native code beforehand

  • Interpreter - interpreted by runtime while running

-> also called a Virtual Machine (VM)

Why does WebAssembly exist?

• Support any language in a minimal runtime

• Completely cross-platform

• Sandboxed by default

• Near-native performance

So how does a wasm app talk to the outside world?

System Interfaces full of System Calls (syscalls)

What does “outside” mean?

• Open files

• Download files

• Open TCP/IP sockets

• Threading

• USB devices

• DOM (webpage)

Legacy system interface: “Emscripten”

• Browser

  • JavaScript API - wasm calls js API of browser to modify DOM, download files, etc.

• Node.js runtime

  • JavaScript API

    • without browser-specific interfaces

    • + server-specific interfaces

-> JavaScript API is inefficient, gigantic, and not secure

WebAssembly System Interface (WASI)

A new syscall interface built for speed, security and cross-platform support.

• Standardised by W3C WASI subgroup

• Built by bytecode alliance

  • With many non-browser partners

    • Cloudflare, Fastly, Siemens, VMware

FAQ: WebAssembly vs Java runtime?

Many similarities both in design and use-cases

• “Write once, run anywhere”

• Architecture-independent bytecode

• JVM on microcontrollers: Johnson Controls heat pumps

• JVM in browser: Java Applets (vSphere web UI)

But ultimately, JVM failed in most of these fields. It remains a single-vendor runtime for a single language family.

Why WebAssembly instead of JVM

WebAssembly learned from the 20+ years of JVM experience.

• Compilation target for all languages, standardized by W3C

  • JVM is too reliant on a single vendor and too focussed on a single language family

  • JVM is too opinionated about languages: e.g. requires classes and garbage collector

• Sandboxed by default with capability-based access to outside world

  • JVM apps have too much access to underlying OS, resulting in security and portability nightmares.

  • JVM assumes all code is trusted

• Software delivery baked-in: Streamability, Hotplugging

  • JVM is too focussed on traditional applications consisting of a single, static, monolithic app already on the user’s machine.