Chapter 07 Operating System Security

OS Security Layers

• User Applications and Utilities

• Operating System Kernel

• Physical Hardware

Deploying System Planning Process must":

• Assess risks and plan the system deployment

• Secure the underlying operating system and then the key applications

• Ensure any critical content is secured

• Ensure appropriate network protection mechanisms are used

• Ensure appropriate processes are used to maintain security

The top four security strategies for prevention are:

1. White-list approved applications

2. Patch third-party applications and OS vulnerabilities

3. Restrict administrative privileges

4. Create a defence-in-depth system

System Security Planning

• First step in deploying a system.

• Aims to maximise security while minimising costs

System Security Planning includes:

• Wide security assessment

• Identification of security requirements (system, apps, data, users)

• Trained and responsible personnel

System Security Planning Process

System Security Planning Steps

1. Define system purpose, stored info, and security needs.

2. Categorize users (e.g., Admins vs. Customers).

3. Choose authentication methods (passwords, biometrics, etc.).

4. Define access controls to data.

5. Assess system's connection to external resources.

6. Assign system administrators and their access methods.

7. Add extra measures (firewalls, antivirus, IDS, logging).

OS Hardening

• Means to configure your OS such that it offers as few vulnerabilities and attack surfaces as possible.

• First critical step in securing a system

OS Hardening Basic Steps

1. Install and patch the OS

2. Harden and configure the OS to adequately address the unidentified security needs of the system

3. Install and configure additional security controls, such as anti-virus, host-based firewalls, and intrusion detection system (IDS)

4. Test the security of the basic operating system to ensure that the steps taken adequately address its security needs

Harden and configure the operating system to adequately address the indentified security needs of the system by:

• Removing unnecessary services, applications, and protocols

• Configuring users, groups, and permissions

• Configuring resource controls

1. Initial Setup and Patching for Operating System

• Begin with secure installation (ideally on a protected network).

• Full installation and hardening process should occur before the system is deployed to its intended location

• Install only what's needed; keep the system minimal.

• The integrity and source of any additional device driver code must be carefully validated

2. Remove Unnecessary Services and Protocols

• Default installations prioritize convenience, not security.

• Only install required packages.

• Fewer services = smaller attack surface.

3. Configure Users, Groups, and Authentication

System planning process should consider:

1. Categories of users on the system

2. Privileges they have

3. Types of information they can access

4. How and where they are defined and authenticated

4. Configure Resource Controls

• Once the users and groups are defined, appropriate permissions can be set on data and resources

• Many of the security hardening guides provide lists of recommended changes to the default access configuration

5. Install Additional Security Controls

Further security possible by installing and configuring additional security tools:

• Anti-virus software

• Host-based firewalls

• IDS or IPS software

• Application white-listing

6. Test the System Security Final step

There are programs specifically designed to:

• Review a system to ensure that a system meets the basic security requirements

• Scan for known vulnerabilities and poor configuration practices

Should be done following the initial hardening of the system.

Final step goal:

• Ensure the previous security configuration steps are correctly implemented

• Identify any possible vulnerabilities

Security Maintenance includes:

• Monitoring and analyzing logging information

• Performing regular backups

• Recovering from security compromises

• Regularly testing system security

• Using appropriate software maintenance processes to patch and update all critical software, and to monitor and revise configuration as needed

Data backup

The process of making copies of data at regular intervals

Data Archive

The process of retaining copies of data over extended periods of time in order to meet legal and operational requirements to access past data

Linux Application and Service configuration

• Generally located either in the /etc dir or in the installation tree for a specific app

• Individual user configs that can override system defaults are located in hidden dot files in each user’s home dir.

Local Exploit

Software vulnerability that can be exploited by an attacker to gain elevated privileges

Remote Exploit

Software vulnerability in a network server that could be triggered by a remote attacker

Logging and log rotation

Relying on default logging configurations in Linux/Unix systems is a common but dangerous oversight. These defaults are typically designed for general performance and minimal disk usage, not for comprehensive security monitoring or forensic analysis.

chroot jail

• Restricts the server’s view of the file system to just a specified portion

• Uses chroot system call to confine a process by mapping the root of the filesystem to some other directory

• File directories outside the chroot jail aren’t visible or reachable

chroot jail disadvantage

Added complexity

Biba Integrity Model

Protects higher-trust system components from being modified by lower-trust processes

Application and service configuration

• Configuration information is centralised in the Registry

• Forms a database of keys and values that may be queried and interpreted by applications

• Registry keys can be directly modified using the “Registry Editor”

• More useful for making bulk changes

Virtualisation

A technology that provides an abstraction of the resources used by some software which runs in a simulated environment called a virtual machine (VM)

Virtualisation Pros

• Better efficiency in the use of the physical system resources

• Provides support for multiple distinct operating systems and associated applications on one physical system

• Raises additional security concerns

Hypervisor

• Software that sits between hardware and the VMs

• Acts as a resource broker

• Allows multiple VMs to coexist

• Each VM includes OS called guest OS

Hypervisor Functions:

1. Execution management of VMs

2. Devices emulation and access control

3. Execution of privileged operations by hypervisor for guest VMs

4. Management of VMs (also called VM lifecycle management)

5. Administration of hypervisor platform and hypervisor software

Types of Hypervisors

• Type 1: Native Virtualisation

• Type 2: Hosted Virtualisation

• Container

Software Defined Networks (SDN)

Enable network segments to logically span multiple servers within and between data centers while using same underlying physical network.

Virtualisation Security Issues

1. Guest OS Isolation

2. Guest OS monitoring by the hypervisor

3. Virtualised environment security

Securing Virtualisation Systems:

1. Carefully plan the security of the virtualized system

2. Secure all elements of a full virtualization solution and maintain their security

3. Ensure that the hypervisor is properly secured

4. Restrict and protect administrator access to the virtualization solution

Hypervisor Security should be:

• Secured using a process similar to securing an operating system

• Installed in an isolated environment

• Configured so that it is updated automatically

• Monitored for any signs of compromise

• Accessed only by authorized administration

Virtualised Infrastructure Security

1. Access to VM image and snapshots must be carefully controlled

2. Access must be limited to just the appropriate guest OSs

3. Systems manage access to hardware resources

3 types of virtual firewalls:

1. VM Bastion Host

2. VM Host-Based Firewall

3. Hypervisor Firewall

VM Bastion Host

Where a separate VM is used as a bastion host supporting the same firewall systems and services that could be configured to run on a physically separate bastion, including possibly IDS and IPS services

VM Host-Based Firewall

Where host-based firewall capabilities provided by the Guest OS running on the VM are configured to secure that host in the same manner as used in physically separate systems

Hypervisor Firewall

Where firewall capabilities are provided directly by the hypervisor