Host and OS Hardening

Host and OS hardening is the process of reducing the attack surface of computers, servers, and other endpoints by applying secure configurations, removing unnecessary services, controlling access, monitoring activity, and managing vulnerabilities.

This topic has four major areas:

Windows security baselines.
Linux security baselines.
EPP and EDR solutions.
Vulnerability management.

A hardened system is not automatically secure forever. It must be continuously monitored, updated, tested, and reviewed.

1. Host and OS Hardening

Concept

A host is any endpoint or system connected to a network, such as:

Desktop computer.
Laptop.
Server.
Virtual machine.
Cloud workload.
Network appliance.
IoT device.

Operating system hardening configures the host to make unauthorized access and exploitation more difficult.

The basic goal is:

Reduce unnecessary exposure, enforce secure settings, limit privileges, and continuously detect weaknesses.

Common hardening activities

Remove unnecessary software.
Disable unused services and ports.
Apply security patches.
Use strong authentication.
Enforce least privilege.
Restrict administrative access.
Enable host-based firewalls.
Configure secure logging.
Encrypt storage and sensitive communications.
Install endpoint protection.
Disable insecure protocols.
Apply configuration baselines.
Back up important data.
Regularly scan for vulnerabilities.

Example

A newly installed Linux server may have multiple services enabled by default. Hardening could involve:

Removing unnecessary packages.
Disabling unused services.
Allowing SSH only from the management network.
Disabling direct root login.
Requiring SSH keys and multifactor authentication.
Enabling a host firewall.
Applying security updates.
Monitoring authentication and system logs.

2. Security Baselines

Concept

A security baseline is a documented set of recommended configuration settings for an operating system, application, device, or service.

Baselines provide a consistent standard for determining whether systems are securely configured. Microsoft describes a security baseline as a group of recommended configuration settings that also explains their security implications.

A baseline may specify:

Password requirements.
Account lockout settings.
Audit policies.
Firewall configuration.
Encryption settings.
Access-control rules.
Logging requirements.
Application restrictions.
Network protocol settings.
User privilege assignments.
Automatic update behavior.

Why baselines are important

Without a baseline, administrators may configure systems inconsistently. One server may have strong security settings while another has weak or outdated configurations.

Baselines help organizations:

Standardize security.
Reduce configuration errors.
Support compliance.
Speed up system deployment.
Detect configuration drift.
Improve auditability.
Create measurable security requirements.

Baseline sources

Common sources include:

Microsoft Security Baselines.
CIS Benchmarks.
DISA STIGs.
NIST guidance.
Vendor hardening guides.
Organizational security policies.

A baseline should not be applied blindly. It should be tested because overly restrictive settings can break applications, affect performance, or prevent legitimate administrative work.

3. Windows Hardening

Common Windows controls

Account security

Disable or rename unnecessary local administrator accounts.
Use separate standard and administrative accounts.
Enforce strong passwords.
Enable account lockout policies.
Use multifactor authentication.
Restrict interactive logon rights.
Remove inactive accounts.

Group Policy

Group Policy allows administrators to centrally configure Windows computers in a domain environment.

It can control:

Password policies.
Windows Defender settings.
Firewall rules.
Audit policies.
Application restrictions.
Device control.
User permissions.
Security options.

For example, an organization may use Group Policy to prevent standard users from installing unauthorized software.

Windows Defender Firewall

The host firewall should restrict unnecessary inbound connections. Outbound traffic may also be controlled where appropriate.

A server should allow only the ports required for its role. For example, a web server may need TCP port 443 for HTTPS but should not expose administrative services to the entire internet.

BitLocker

BitLocker encrypts Windows storage volumes. It helps protect data if a laptop or drive is stolen.

Encryption does not prevent malware from using a system while it is running. It mainly protects data at rest.

User Account Control

User Account Control, or UAC, helps prevent applications from silently performing administrative actions. It prompts the user when elevated privileges are requested.

PowerShell security

PowerShell is useful for administration but can also be abused by attackers. Organizations can improve security by:

Restricting unnecessary PowerShell access.
Enabling PowerShell logging.
Monitoring suspicious scripts.
Applying application-control policies.
Using constrained language mode where appropriate.
Blocking unauthorized macros and script interpreters.

Security auditing

Windows auditing can record:

Successful and failed logons.
Account changes.
Privilege use.
Process creation.
Policy changes.
Object access.
PowerShell activity.

These events can be forwarded to a central logging or SIEM system.

Windows baseline workflow

Identify the Windows edition and role.
Select an appropriate baseline.
Test the baseline in a lab or pilot group.
Compare current settings with the baseline.
Document approved exceptions.
Deploy settings through Group Policy, Intune, or another management platform.
Validate the result.
Monitor for configuration drift.
Review the baseline when the operating system or business requirements change.

Microsoft provides the Security Compliance Toolkit to help administrators download, analyze, test, edit, and store recommended security configurations.

4. Linux Hardening

Linux hardening varies between distributions such as Ubuntu, Debian, Fedora, Rocky Linux, and other enterprise systems. The exact commands and configuration files may differ, but the security principles are similar.

Common Linux controls

User and privilege management

Disable unnecessary accounts.
Use sudo instead of direct root login.
Require strong authentication.
Use SSH keys where appropriate.
Restrict administrative commands.
Review membership in privileged groups.
Remove inactive users.

SSH security

SSH is commonly used for remote administration. Recommended controls may include:

Disable direct root login.
Disable password authentication when key-based access is practical.
Restrict SSH to management networks.
Use multifactor authentication.
Apply connection rate limits.
Monitor failed login attempts.
Use modern cryptographic algorithms.

Example configuration concepts:

PermitRootLogin no PasswordAuthentication no AllowGroups administrators

These settings must be tested carefully because an incorrect SSH configuration can lock out administrators.

Services and ports

Administrators should identify listening services and disable those that are not required.

Useful activities include:

Listing active services.
Reviewing listening ports.
Removing unused packages.
Disabling legacy protocols.
Restricting services with firewall rules.

For example, a database server should not expose its database port to the public internet unless there is a specific, justified requirement.

File permissions

Linux permissions should follow least privilege. Sensitive files should not be writable by everyone.

Important areas include:

System configuration files.
SSH keys.
Application secrets.
Log files.
User home directories.
Backup files.
Database files.

Special permissions such as SUID and SGID should be reviewed because they can allow programs to run with elevated privileges.

Mandatory access control

Linux systems may use:

SELinux.
AppArmor.

These systems add policy-based restrictions that limit what applications can access, even when a process is compromised.

Logging

Important Linux logs may record:

Authentication attempts.
Privilege escalation.
Service activity.
Kernel events.
Firewall events.
Application errors.

Logs should be protected from unauthorized modification and forwarded to a central system when possible.

Updates

Linux systems should receive:

Kernel updates.
Security patches.
Package updates.
Application updates.
Firmware updates where applicable.

Updates should be tested and tracked, especially on production servers.

Linux baseline workflow

Identify the Linux distribution and version.
Determine the system’s role.
Select a suitable CIS Benchmark, STIG, or organizational baseline.
Scan the system.
Review failed controls.
Test remediation.
Apply approved changes.
Rescan the system.
Document exceptions.
Repeat the assessment periodically.

OpenSCAP and SCAP Security Guide profiles can be used to assess and remediate Linux systems against selected security profiles.

5. EPP and EDR Solutions

Endpoint Protection Platform

An Endpoint Protection Platform, or EPP, protects endpoints from known and common threats. It usually combines traditional antivirus capabilities with additional prevention features.

EPP functions may include:

Malware detection.
Real-time file scanning.
Web protection.
Email protection.
Exploit prevention.
Ransomware protection.
Device control.
Application control.
Quarantine.
Signature and reputation checks.

EPP mainly focuses on preventing malicious activity before or during execution.

Endpoint Detection and Response

Endpoint Detection and Response, or EDR, continuously monitors endpoint activity and helps security teams investigate and respond to suspicious behavior. CISA describes EDR as providing monitoring and control of endpoint devices across detection, response, recovery, and follow-up analysis.

EDR may collect:

Process activity.
Command-line arguments.
File creation and modification.
Registry changes.
Network connections.
User logons.
PowerShell activity.
Persistence mechanisms.
Security alerts.
Parent-child process relationships.

EPP versus EDR

Feature	EPP	EDR
Main purpose	Prevent common threats	Detect, investigate, and respond to threats
Primary focus	Prevention	Visibility and response
Typical detection	Malware signatures, reputation, behavior	Suspicious activity chains and attacker behavior
Data collection	Usually focused on security events	Detailed endpoint telemetry
Response	Quarantine or block files	Isolate host, terminate process, remove persistence
Best example	Blocking a known malware file	Investigating an attacker using PowerShell and stolen credentials

Many modern products combine EPP and EDR capabilities.

EDR example

An attacker sends a malicious document to an employee. The sequence may be:

The user opens the document.
The document launches an unusual script.
The script starts PowerShell.
PowerShell downloads a payload.
The payload creates a scheduled task.
The system connects to an unfamiliar external address.

An EDR platform can correlate these events into a single suspicious activity chain. Security personnel may then:

Isolate the endpoint.
Stop malicious processes.
Remove persistence.
Block the external address.
Identify other affected systems.
Review stolen credentials.
Preserve evidence for investigation.

EPP and EDR limitations

They may produce too many alerts.
Encrypted or fileless attacks can be difficult to detect.
Attackers may attempt to disable the agent.
Poorly configured exclusions can create blind spots.
Detection does not guarantee successful response.
Endpoint telemetry may raise privacy and data-retention concerns.
Unsupported operating systems may have limited protection.

EDR is most effective when combined with trained analysts, incident-response procedures, identity security, patching, and centralized logging.

6. Vulnerability Management

Concept

Vulnerability management is a continuous process for identifying, assessing, prioritizing, remediating, and verifying security weaknesses.

A vulnerability may exist in:

Operating systems.
Applications.
Libraries.
Firmware.
Network devices.
Cloud services.
Configurations.
User accounts.
Exposed services.

Vulnerability management lifecycle

1. Discover assets

Create and maintain an inventory of:

Workstations.
Servers.
Virtual machines.
Applications.
Network devices.
Cloud resources.
IoT devices.
Software versions.
System owners.

You cannot reliably secure assets that are unknown or unmanaged.

2. Scan for vulnerabilities

Scanning tools compare systems against vulnerability databases, software versions, configuration rules, and security checks.

Scanning types include:

Authenticated scanning.
Unauthenticated scanning.
Network scanning.
Web application scanning.
Cloud configuration assessment.
Container image scanning.
Dependency scanning.

Authenticated scanning generally provides better visibility because the scanner can inspect installed software and local configuration.

3. Validate findings

Not every scanner result is equally reliable. Security teams should confirm:

Whether the affected software is actually installed.
Whether the vulnerability applies to the system.
Whether compensating controls exist.
Whether the finding is a false positive.
Whether exploitation is possible in the environment.

4. Prioritize risk

Vulnerabilities should not be prioritized only by severity score. Risk also depends on:

Whether the asset is internet-facing.
Whether exploitation is active.
Whether exploit code is publicly available.
Business importance.
Data sensitivity.
Exposure to attackers.
Availability of a patch.
Existing compensating controls.

A medium-severity flaw on an internet-facing payment server may deserve faster action than a critical flaw on an isolated test machine.

5. Remediate

Remediation may involve:

Applying a patch.
Upgrading software.
Removing vulnerable software.
Changing configuration.
Disabling an exposed service.
Restricting network access.
Replacing unsupported systems.
Applying a temporary workaround.

6. Verify

After remediation, rescan or test the system. A ticket should not be closed merely because someone reports that a patch was installed.

7. Report and improve

Track:

Number of open vulnerabilities.
Age of vulnerabilities.
Mean time to remediate.
Recurring vulnerabilities.
Unmanaged devices.
Patch compliance.
Exceptions and their expiration dates.

Vulnerability versus threat versus risk

Term	Meaning	Example
Vulnerability	A weakness that can be exploited	An unpatched web server
Threat	A potential cause of harm	A cybercriminal group
Exploit	A method or code used to abuse a weakness	Malicious request targeting the server
Risk	The possibility and impact of harm	Data theft caused by exploiting the server
Control	A measure that reduces risk	Patch, firewall rule, or segmentation

Vulnerability management example

Suppose a vulnerability scanner identifies an outdated web framework on an internet-facing server.

A security team may:

Confirm the framework version.
Determine whether the server handles sensitive data.
Check whether the vulnerability is actively exploited.
Apply the vendor patch in a test environment.
Schedule production deployment.
Temporarily restrict access if patching is delayed.
Rescan the server.
Document the result and close the ticket only after verification.

7. How the Controls Work Together

Host hardening, endpoint protection, and vulnerability management support one another.

text

Secure Baseline
      |
Hardening and Configuration Control
      |
EPP Prevention
      |
EDR Monitoring and Response
      |
Vulnerability Scanning
      |
Patch and Remediation
      |
Continuous Verification

Example scenario

An organization runs a Linux application server and Windows employee laptops.

The Linux server is configured according to a CIS or STIG-based baseline.
SSH is restricted to the management network.
Unused services are disabled.
The server is scanned for vulnerabilities.
EDR monitors process and network behavior.
Windows laptops use Microsoft security baselines and host firewalls.
EPP blocks malware.
EDR detects suspicious PowerShell activity.
A vulnerability management platform identifies missing patches.
Critical internet-facing vulnerabilities receive priority.
Management receives reports about patch compliance and exceptions.

Hardening reduces the number of possible attack paths. EPP attempts to prevent malicious activity. EDR identifies suspicious behavior and supports response. Vulnerability management finds weaknesses before attackers exploit them.\

8. Common Implementation Mistakes

Applying a baseline without testing business applications.
Giving users permanent local administrator rights.
Leaving unused services enabled.
Using unsupported operating systems.
Installing endpoint protection but ignoring its alerts.
Creating excessive antivirus exclusions.
Scanning only the external network.
Failing to inventory cloud and IoT assets.
Treating every vulnerability as equally urgent.
Closing remediation tickets without verification.
Allowing exceptions to remain permanently.
Collecting EDR data without defining retention and privacy rules.
Failing to monitor whether security agents are running.
Relying only on antivirus signatures.
Neglecting backup and recovery testing.