4.9 Given a scenario, use data sources to support an investigation
Networks, hosts, and applications generate vast amounts of data through various mechanisms.
A significant challenge for organizations is identifying these data sources and scanning them for threat indicators.
Security professionals must efficiently identify and utilize appropriate data sources to perform incident response.
Data Sources, Dashboards, and Reports
Definition of Data Source in Incident Response
A data source refers to any entity that can undergo analysis to uncover indicators in digital forensics or incident response investigations.
Types of Data Sources Used in Investigations
System memory and media device file systems: Involves retrieving data and metadata from the system memory and file systems.
Log files from network appliances: Collected from devices such as switches, routers, and firewalls/UTMs.
Captured network traffic: This includes monitoring by sensors or alerts from intrusion detection systems.
Logs from network-based vulnerability scanners: These logs contain alerts and information regarding vulnerabilities.
Operating system logs: Generated by the client and server OS components.
Host application logs: Created by various applications and services running on hosts.
Endpoint security software logs: These log files originate from host-based intrusion detection systems, antivirus, and firewall software.
Challenges Posed by Data Source Diversity
The vast diversity and volume of data present significant challenges to forensic investigations.
SIEM (Security Information and Event Management) tools are utilized to aggregate and correlate multiple data sources, serving as a central point for dashboards and automated reporting.
Challenges related to data are often categorized into the "Vs":
Volume: The quantity of data.
Velocity: The speed of data generation.
Variety: The different forms of data.
Veracity: The accuracy and trustworthiness of the data.
Value: The usefulness of the data in decision-making.
Event Dashboards
Definition and Functionality
An event dashboard serves as a console for managing day-to-day incident responses. It summarizes information from underlying data sources for task support.
Types of Dashboards
Dashboards can be tailored for various roles:
Incident Handler's Dashboard: Displays uncategorized events assigned to the handler's account and includes visualizations of key metrics.
Manager's Dashboard: Presents overall status indicators, such as the number of unclassified events for all handlers.
Example
Default dashboard console in Security Onion provides an overview of event types and volumes.
Automated Reports
Reporting Types Available in SIEM
Alerts and Alarms: Designed to detect threat indicators in data and can initiate incident cases; a significant aspect of an analyst's daily tasks.
Status Reports: Communicates threat levels or incident numbers and evaluates security effectiveness, often necessary for management decision-making.
Reports are customizable to meet the audience's objectives; irrelevant data can hinder identification of remediation actions.
Log Data
Importance of Log Data
Log data plays a critical role in investigating security incidents. It is essential to understand various log file types to support investigation scenarios.
Log File Composition
Event data: Generated by processes on network appliances and hosts, written to log files or databases.
Components of an Event:
Message data: The event notification or alert raised by a process (e.g., "Login failure").
Metadata: Includes source, time, and categorization details.
Examples of Log Formats
Windows Event Viewer: Structured format logging that includes headers with source, level, user, timestamp, category, and keywords.
Syslog: An open format and protocol used for logging messages across various host types (e.g., devices like switches, routers, firewalls, UNIX/Linux servers).
Accurate Logging Requirements
Hosts must be time-synchronized to ensure accurate logging, ideally using Universal Coordinated Time (UTC) to avoid discrepancies.
Syslog Message Structure
Components of a Syslog Message
PRI code: A primary code derived from facility and severity level.
Header: Contains timestamp, host name, app name, process ID, and message ID.
Message part: Displays the source process along with the associated content.
Message formatting can vary based on the application requirements, often involving space- or comma-delimited fields.
Aggregation of Log Data
Organizations seek better visibility through SIEM solutions that aggregate logs, providing a unified view of network hosts and appliances.
Log Collection Methods:
Agent running on each host.
Syslog or similar protocols for forwarding event data.
Operating System (OS) Logs
Types of OS Logs
OS maintains various logs capturing events during user and software interactions:
Different logs correspond to various functionalities (e.g., security logs for audit events).
Audit events typically categorized as success/accept or fail/deny.
Categories of Host Operating System Logs
Authentication Events: Records user login attempts and special privilege requests.
File System Events: Monitors permissions for file read or modification attempts.
File system auditing usually requires explicit configuration due to potential data volume.
Main Windows Event Logs
Application Log: Events from application processes like crashes.
Security Log: Audit logs for failed logins and file access denials.
System Log: Records events from the OS kernel processes and services.
Linux Logs
Implementation of Linux Logging
Logging mechanisms may vary by distribution:
Some distributions utilize syslog to store subsystem messages, while others might implement Journald.
Journald messages can be read with
journalctl, and some can be exported to text files via syslog.
Key Linux Log Files
/var/log/messages or /var/log/syslog: Central location for system-generated events.
/var/log/auth.log (Debian/Ubuntu) or /var/log/secure (RedHat/CentOS/Fedora): Tracks user authentication data.
Package Manager Logs: Maintains records of software installations and updates.
macOS Logs
Unified Logging System in macOS
macOS logs can be accessed through the Console app or using the
logcommand,Filtering options available to review security-related events, including logins and installations.
Application and Endpoint Logs
Application Logs
Managed by applications themselves rather than the OS.
Windows Event Viewer**: Contains application-specific logs that can be written by any authenticated account.
Application developers determine whether their software logs data via Event Viewer or in a custom format.
Endpoint Logs
These logs record events monitored by security software rather than the host OS.
May include details from firewalls, intrusion detection systems, vulnerability scanners, and antivirus products.
Integrated platforms may refer to EPP (Endpoint Protection Platform), EDR (Endpoint Detection and Response), or XDR (Extended Detection and Response).
Summary from Endpoint Protection Logs
Analysis of endpoint logs can provide insights on malware detection, intrusion events, or patch management status.
Vulnerability Scans
Role of Vulnerability Scanners
Generating summary reports as well as logging each detected vulnerability within a SIEM for detailed retrieval.
These logs identify missing patches and noncompliance with security standards, aiding in baseline security assessments.
Network Logs
Generation of Network Logs
Created by appliances like routers, firewalls, switches, and access points. Captures operational and traffic/access records.
Types of Data Insights from Network Logs
Switch Logs: Detect incidents where multiple MAC addresses are used, indicating potential on-path attacks.
Firewall Logs: Identify scanning attempts on blocked ports and record events based on configured rules.
Access Point Logs: Trace disassociation events indicative of wireless network threats.
Firewall Logs
Configuration and Event Types
Firewall rules can log actions triggered during packet traffic analysis:
Timestamp, interface, rule matched, and traffic direction (incoming/outgoing) are recorded.
Packet information (source and destination IP addresses and ports) is included.
IDP/IPS Logs
Events logged upon matching traffic patterns with predefined rules, generating potentially large volumes of data.
Configurable to log only significant events, similar to firewalls, helping in detecting intrusion attempts based on specific rules.
Packet Analysis
Definition of Packet Analysis
Refers to an in-depth examination of network traffic using tools like Wireshark.
Involves decoding packet structures to reveal header information and payload.
Insights from Network Traffic Analysis
Extracting insights can reveal potential security breaches; traffic is typically analyzed based on triggering firewall/IDS rules.
Retrospective Network Analysis (RNA): Enables recording of all network events and detailed packet contents.
Applications of Packet Analysis
Identifying manipulated packets or tracing data exfiltration attempts.
Analyzing packet contents can yield information about attack tools used and assist in extracting malware for further analysis.
Metadata
Overview of Metadata
Defines properties of data utilized by applications, stored on media, or transmitted over networks.
Useful in establishing timelines during investigations and revealing additional evidence.
Types of Metadata
File Metadata: Attributes such as creation, access, modification times, and permissions.
Web Metadata: Server response headers contain information on resource types and authorization aspects.
Email Metadata and Headers
Email headers provide transmission details, including sender, recipient, and intermediary servers.
Headers can disclose spam checks and other changes made by MTAs during transmission.
Tools for Analyzing Email Headers
Tools like the Message Analyzer assist in clarifying the paths and information encapsulated within email headers.
Example of Analyzing Headers
Examining headers in phishing messages reveals techniques like typosquatting, where similar names are used to deceive recipients.