Virtualization Notes

Virtualization Essentials - Notes

Chapter 1: Understanding Virtualization

  • Virtualization Defined:
    • Virtualization is a disruptive technology that changes how physical computers are handled, services are delivered, and budgets are allocated.
    • It involves abstracting a physical component into a logical object to increase the utility of the resource.
    • Virtualization of entire computers is the primary focus.
  • Virtual Machine (VM) Definition:
    • A VM can virtualize all hardware resources, including processors, memory, storage, and network connectivity.
  • Virtual Machine Monitor (VMM) / Hypervisor:
    • Software that provides the environment in which VMs operate.
  • Popek and Goldberg’s VMM Requirements:
    • Fidelity: The VM environment is essentially identical to the original physical machine.
    • Isolation/Safety: The VMM must have complete control over system resources.
    • Performance: Little to no performance difference between the VM and a physical equivalent.
  • Microsoft Windows Impact on Server Growth:
    • Windows became the dominant PC operating system.
    • Businesses increasingly relied on computers for daily operations.
    • Windows servers became capable of hosting more powerful applications, previously run on minicomputers and mainframes.
    • Challenges arose because Windows was initially designed for single-user operation.
    • The "one server, one application" best practice emerged to avoid resource contention and operating system failures.
  • Corporate Politics and Redundancy:
    • Departments within companies often preferred dedicated infrastructure for data control.
    • This led to redundant applications and increased capital costs.
  • Hardware Cost Reduction:
    • Business demand, competition, Moore’s law, and technological improvements drastically reduced hardware costs.
    • This made it more affordable for departments to manage their IT infrastructure.
  • Data Centers:
    • Data centers evolved from specialized computer rooms to entire buildings dedicated to computing infrastructure.
    • These centers require significant power, cooling, cabling, and security systems.
  • Internet Bloom:
    • The Internet's rise drove businesses to deploy even more servers for web-enabled applications.
    • Companies like Amazon and Google created disruptive business models dependent on large server farms.
  • Environmental Impact:
    • Between 2000 and 2006, energy use by U.S. data centers doubled, with expectations of doubling again in the next five years.
    • Servers consumed about 2% of the total electricity, with cooling consuming an equal amount.
  • Data Center Challenges:
    • Data centers faced physical limits, running out of space, power, and cooling capacity.
    • Lost servers (running servers with unknown ownership or purpose) were a common issue.
  • Moore’s Law:
    • Moore’s law states that processing power roughly doubles every 18 months.
    • This applies to processing power, memory capacity, and digital camera megapixels.
  • Impact of Moore’s Law on Server Replacement:
    • Servers are routinely replaced every 3 to 5 years.
    • Newer servers are significantly more powerful than older ones due to Moore’s Law.
  • Server Utilization:
    • Server workloads often don’t scale at the same rate as server capabilities.
    • Headroom (extra capacity) in servers increased substantially over time.
    • Data centers often had low average server utilization (10-15%), with many servers operating at less than 5% utilization.
  • Virtualization to the Rescue:
    • Virtualization enables multiple operating system workloads to run on the same server hardware, isolated from each other.
    • VMware provided the first commercially available virtualization solution for x86 computers in 2001.
    • Xen, an open-source offering, arrived two years later.
  • Server Consolidation:
    • Virtualization allows condensing multiple physical servers into fewer servers running many virtual machines.
    • This increases the utilization rate of physical servers.
    • Consolidation ratio is the number of VMs on a server (e.g., 8 VMs = 8:1 consolidation ratio).
  • Cost Reduction through Consolidation:
    • Total cost of ownership for a server is 3-10 times the server cost over three years.
    • Consolidation reduces hardware maintenance, power, and system administration costs.
  • Containment:
    • Companies began virtualizing server workloads instead of purchasing new hardware.
    • This reduced the need for hardware refreshes and lowered management costs.
  • Consolidation Ratio improvements:
    • First x86 hypervisors produced consolidation ratios of 5:1.
    • More powerful chips and larger memory allowed ratios of 10:1 to 100:1.
  • Virtualization Implementation Course:
    • Initial focus: infrastructure services and older servers.
    • Virtualization-first policies: new projects utilize virtual resources instead of physical hardware.
    • Test and development servers are moved to virtual infrastructure.
  • Migration of Large Applications:
    • Larger applications often require larger hardware and specialized operating systems.
    • Linux can run the same application software as vendor-specific hardware.
    • Virtualization of Linux leads to a more flexible, less expensive environment.
  • Virtualization for Availability:
    • Virtual servers, encapsulated as files, enhance availability.
    • Virtual machines can be moved between physical hosts without interruption.
    • Resources (processors, memory) can be added without rebooting.
    • Entire data centers can be restored quickly in case of disasters.
  • Virtualization for Tier-One Applications:
    • Remaining physical servers running strategic business applications are virtualized.
    • This provides greater availability, agility, flexibility, and manageability.
  • Virtualization and Cloud Computing:
    • Virtualization is the foundation for cloud computing.
    • It transforms data centers into self-managing, highly scalable pools of resources.
    • It enables the concept of a virtual data center, offering resources on an as-needed basis.
  • Virtualization for Automation:
    • Virtualization provides greater automation opportunities.
    • It reduces administrative costs and increases the ability to dynamically deploy solutions.
  • Hyperconverged Infrastructure:
    • Hardware vendors create servers optimized for hypervisors, containing computer, networking, and storage resources.
    • These prebuilt blocks allow rapid scalability and reduce the effort to start or expand a virtual environment.
    • This model has expanded out to vendors offering specialized areas (Oracle Exadata, IBM PureSystems)
  • Virtualization Software Operation Models:
    • Server Virtualization: Hypervisors abstract the physical layer and present it to virtual machines.
    • Desktop Virtualization: Virtual desktops run on servers in the data center with access through thin clients.
    • Application Virtualization: Applications are virtualized, packaged, and delivered to users.
  • Virtualizing Servers (Hypervisors):
    • Hypervisors are installed directly onto servers and act as an interface between hardware and virtual machines.
    • They enable enhanced availability and improve provisioning and management.
  • Virtual Machines in Server Virtualization:
    • Virtual machines contain everything their physical counterparts do (operating systems, applications) but are packaged in data files.
    • This makes them more flexible and manageable.
  • Virtual Desktop Advantages:
    • Virtual desktops run on more powerful and reliable servers than traditional PCs.
    • Applications are located in the same data center, reducing network traffic.
  • Virtual desktop lifecycle advantages:
    • Thin clients are more reliable and use less electricity than PCs.
    • If a thin client breaks, it can be replaced without data loss.
    • Desktop images can be shared among many users and managed centrally.
    • New methods of protection with virtual appliances.
  • Application Virtualization
    • Two main reasons for application virtualization are ease of deployment, and application conflicts.
    • Encapsulate the entire program and avoid affecting other applications.
  • Containers
    • Larger software package and include a shared copy of a single operating system for many workloads to leverage.
  • Impact to Architectural and Personnel Business side
    • Virtual machines have now require virtualization administrators to manage those virtual infrastructures, replacing the traditional silos.
    • Virtualization administrators now needs access into the legacy IT silos of networking, storage, and server administration.

Chapter 2: Understanding Hypervisors

  • Hypervisor Definition:
    • A hypervisor is the single arbiter of physical resources, essentially software that sits between physical server resources and the virtual machines.
    • It provides a virtual environment, virtual networks, and clustering for high availability.
  • Legacy:
    • Evolved from Virtual Machine Managers (VMM) in IBM mainframe, used to solve issues by managing memory resources
    • Allows greater and efficient use of faster processors and higher memory offerings
    • Software layer residing above hardware and below virtual machines.
  • 1990’s: Researchers started investigate the possibility for an commercially affordable version of a VMM.
    Challenges:
    Provide a guest operating system without any modifications
  • Virtual Machine Monitor Structure:
    • Software layer between the host (hardware) and virtual machines (guests).
  • Hypervisor Types:
    • Type 1: Runs directly on server hardware (bare-metal).
    • Type 2: Runs as an application on top of a traditional operating system.
  • Type 1 Hypervisors:
    • Communicate directly with hardware resources, making them more efficient than Type 2.
    • Considered more secure as guest operations are isolated.
      *Virtual machine can damage itself but isolated from other guests and the hypervisor.
    • Require less processing overhead and do not require the cost of a host OS.
  • Type 2 Hypervisors:
    • Run atop a traditional operating system.
    • Support a large range of hardware due to operating system compatibility.
    • Easy to install and deploy.
    • Less efficient due to the extra layer between hypervisor and hardware.
    • Less reliable with more points of failure because an underlying operating system can affect the VM’s operation.
  • Roles and Characteristics Illustrated by Popek and Goldberg
    • Provide virtual resources on physical resources
      Two thing has to happen:
      Holodecks: Make the Host see and access the Hardware Resources
      Traffic Cone: Balance Workload
  • Hardware Resources Abstracted
    • Disk Drives
    • Memory
    • Make Network Calls
  • Resource Allocation
    • Requests Hand off to Hypervisor
    • Requests Service in an manner. Manage and allocate more resources to ensure that
      the configuration is adequate.
  • Vendor and Operating System Specific Solutions
    • Oracle Solaris Zones
    • BSD Jails
    • PowerVM on IBM AIX
  • Alternatives Operating System Virtualization
    • Virtuozzo
    • Docker
  • VMware ESX:
    • Founded in 1998:
      First company to develop commercially for x86 virtualization solution
    • VMware workstation first release: 1999
      Created developers to run on desktop
  • VMware ESX 1.0 and GSX 1.0
    • Both released in 2001
    • ESX was a Type 1 Hypervisor
    • GSX was a Type 2 Hypervisor
    • GSX was renamed VMware Server but is no longer available
      *VMware still holds close to 70 percent of the market.
  • VMware ESXi Achitecture
    • VMkernel contains all of the necessary processes to support virtual machines and manage the hardware and availability resources.

ESX vs ESXi:
* VMware decided that this model was not sustainable due to the service of roughly being 30 times the size of the hypervisor. The second Reason was security.
* Linux is a well-understood environment, and there was concern that the hypervisor could be compromised through the service console.
*VMware released two versions, classic ESX and ESXi, from version 3.5 in 2007 through version 4.1 in 2010. As of the 2011 release 5, only the ESXi architecture is available.

  • VMware Features and Capabilities
    • Transparent page sharing and memory ballooning facilitate efficient memory usage.
    • High availability and fault tolerance enhance uptime without additional software.
  • Citrix Hypervisor (Xen)
    • Began as research in the late 1990;s Cambridge university
    • Create and efficient platform for distributed computing.
    • In 2002, the code was make an open-source project, allowing to improve other features.
    • XenSource was founded in 2004 to brint to marked, but the open-source project still remained open.
    • RedHat, Novell, SUm all added Xen hypervisor to the product offerings, bringing it to the mainstream.
    • Other soltuions are, such as Oracle VM and Amazon Web Service, also used Xen as their virtualization framework.
      Achitecture:
      The hypervisor is a bare-metal solution and sits directly on the hardware, but the implementation shows some differences from the VMware architecture.
      Has Domain 0: Special Guest, also referred to as Dom0
      This guest gets booted when the hypervisor is booted, and it has management privileges different from
      the other guests. Because it has direct access to the hardware, it handles all of the
      I/O for the individual guests. It also handles the hardware device driver support.
      With more request the Hypervisor

Up to the Dom0 Guest. This impacts availabilty because Dom0 resources must remain available as well.