SIEM Logging: Overview & Best Practices
Security Information and Event Management (SIEM) is a pivotal cybersecurity tool that centralizes the security
information swirling around the thousands of endpoints, servers, and applications within your organization. As
end-users and devices interact with every application touchpoint, they leave digital fingerprints in the form of
logs. These files have traditionally played a large role in bug fixing and quality control: after all, they provide
error info straight from the source.
In 2005, however, security professionals began to realize the true potential that lay in these small files. They
provide a host of real-time data that can be fed into SIEM logging that monitors such IT infrastructure. Ever
since then, the tradeoff between threat visibility and event log volume has been carefully trodden by security
professionals. This article will cover several best practices for SIEM log management – with which your security
tooling can reach its full potential
Next Generation SIEM
Stellar Cyber Next-Generation SIEM, as a critical component within the Stellar Cyber Open XDR Platform,...
Experience AI-Powered Security in Action!
Discover Stellar Cyber's cutting-edge AI for instant threat detection and response. Schedule your demo today!
Why SIEM Matters
The main significance of SIEM log management lies in its ability to efficiently analyze vast quantities of these
logs, enabling security analysts to focus on critical threats. Moreover, SIEM systems normalize data in
heterogeneous enterprise environments for simplified analysis, provide real-time and historical threat analysis
based on log data, send automated alerts prioritized by severity when potential security threats are detected,
and maintain detailed records crucial for incident response and forensic investigations.
In essence, SIEM log management is imperative for establishing and sustaining a robust and responsive security
posture in the intricate landscape of contemporary IT environments.
What Is SIEM Logging and How Does it Work?
In order to provide real-time security, SIEM software gathers logs from multiple sources and transmits them to a central logging system. With ‘What is SIEM?’ answered, it’s possible to dig deeper into the varied methods employed by SIEM tooling
Agent-based Log Collection
This log aggregation happens at the local level. The agents are imbued with log filters and normalization
capabilities, allowing for greater resource efficiency. Agents generally take up less network bandwidth, thanks
to the fact that log data is compressed into batches.
Direct Connection
Agentless logging – often facilitated by network protocols or API calls – is another form of SIEM logging that
sees the SIEM program retrieve log files directly from storage, often in syslog format. The benefits range from
ease of deployment, to eliminating the need for software or version updates – this option often contributes to
reduced SIEM maintenance costs.
Event Streaming Protocols
While both agent-based and agentless logging methods offer distinct ways to collect data, event-based
architecture re-conceptualizes this process as flows of events traveling through a river. Each event can be
captured and further processed by downstream consumers. NetFlow, a protocol devised by Cisco, is one
example of this approach. It gathers IP network traffic whenever an interface is entered or exited. The analysis
of NetFlow data empowers network administrators to discern critical information, including the source and
destination of traffic, the protocols utilized, and the duration of communication. This data is collected through
a NetFlow collector, which not only captures the essential traffic details but also records timestamps, requested
packets, and the entry and exit interfaces of the IP traffic.
In the face of increasingly sophisticated attacks, event streaming plays a crucial role by funneling comprehensive information about network traffic to security devices, including next-generation firewalls (NGFW), intrusion detection and prevention systems (IDS/IPS), and security web gateways (SWG).
Overall, SIEM logging emerges as a pivotal element in modern cybersecurity, offering both real-time and historical threat analysis based on log data. However, it’s vital to keep in mind the differences between plain old log management and SIEM.
In the face of increasingly sophisticated attacks, event streaming plays a crucial role by funneling comprehensive information about network traffic to security devices, including next-generation firewalls (NGFW), intrusion detection and prevention systems (IDS/IPS), and security web gateways (SWG).
Overall, SIEM logging emerges as a pivotal element in modern cybersecurity, offering both real-time and historical threat analysis based on log data. However, it’s vital to keep in mind the differences between plain old log management and SIEM.
SIEM vs Log Management: Key Differences
While logs form the backbone of SIEM capabilities, there’s a key difference between the processes of SIEM and
log management. Log Management involves the systematic collection, storage, and analysis of log data sourced
from various channels. This process offers a centralized perspective on all log data and is predominantly
employed for purposes such as compliance, system troubleshooting, and operational efficiency. However, log
management systems do not inherently conduct analysis on the log data – rather, it’s up to the security analyst
to interpret this information and judge the validity of potential threats.
SIEM takes this process one step further by cross-referencing event logs with contextual information pertaining to users, assets, threats, and vulnerabilities. This is achieved through a diverse array of algorithms and technologies for threat identification:
SIEM takes this process one step further by cross-referencing event logs with contextual information pertaining to users, assets, threats, and vulnerabilities. This is achieved through a diverse array of algorithms and technologies for threat identification:
- Event Correlation involves the use of sophisticated algorithms to analyze security events, identifying patterns or relationships indicative of potential threats and generating real-time alerts.
- User and Entity Behavior Analytics (UEBA) relies on machine learning algorithms to establish a baseline of normal activities specific to users and the network. Any deviations from this baseline are flagged as potential security threats, allowing for complex threat identification and the detection of lateral movement.
- Security Orchestration and Automation Response (SOAR) enables SIEM tools to automatically respond to threats, eliminating the need to wait for a security technician to review alerts. This automation streamlines incident response and is an integral component of SIEM.
- Browser Forensics and Network Data Analysis utilize SIEM’s advanced threat detection capabilities to identify malicious insiders. This involves examining browser forensics, network data, and event logs to reveal potential cyber attack plans.
Accidental Insider Attack
An example of how each component can be put into practice is an accidental insider attack.
These attacks occur when individuals inadvertently assist external malicious actors in advancing during an attack. For instance, if an employee were to misconfigure a firewall, it could expose the organization to increased vulnerability. Recognizing the critical importance of security configurations, a SIEM system can generate an event each time a change is made. This event is then elevated to a security analyst for thorough examination, ensuring that the alteration was intentional and correctly implemented, thereby fortifying the organization against potential breaches stemming from unintentional insider actions.
In cases of outright account takeover, UEBA allows for the detection of suspicious activities such as the account accessing systems outside their usual pattern, maintaining multiple active sessions, or making any changes to root access. In the event of a threat actor attempting to escalate privileges, a SIEM system promptly escalates this information to the security team, facilitating swift and effective responses to potential security threats.
These attacks occur when individuals inadvertently assist external malicious actors in advancing during an attack. For instance, if an employee were to misconfigure a firewall, it could expose the organization to increased vulnerability. Recognizing the critical importance of security configurations, a SIEM system can generate an event each time a change is made. This event is then elevated to a security analyst for thorough examination, ensuring that the alteration was intentional and correctly implemented, thereby fortifying the organization against potential breaches stemming from unintentional insider actions.
In cases of outright account takeover, UEBA allows for the detection of suspicious activities such as the account accessing systems outside their usual pattern, maintaining multiple active sessions, or making any changes to root access. In the event of a threat actor attempting to escalate privileges, a SIEM system promptly escalates this information to the security team, facilitating swift and effective responses to potential security threats.
SIEM Logging Best Practices
SIEM plays a pivotal role in an organization’s cybersecurity strategy, but implementing it requires a savvy approach to the logs and correlation rules at the heart of such software.
#1. Select Your Requirements With a Proof of Concept
When trying out a new SIEM tool, Proof of Concepts provide a testing ground. During the PoC phase, it is
crucial to personally direct logs to the SIEM system to gauge the solution’s ability to normalize the data according to specific requirements. This process can be bolstered by incorporating events from non-standard
directories within the event viewer.
This POC is where it’s possible to establish whether agent-based log collection is best for you. If you’re hoping to gather logs over Wide Area Networks (WANs) and through firewalls, using an agent for log collection could contribute to a reduction in server CPU utilization. On the other hand, agentless collection can relieve you of software installation demands, and result in lower maintenance costs.
This POC is where it’s possible to establish whether agent-based log collection is best for you. If you’re hoping to gather logs over Wide Area Networks (WANs) and through firewalls, using an agent for log collection could contribute to a reduction in server CPU utilization. On the other hand, agentless collection can relieve you of software installation demands, and result in lower maintenance costs.
#2. Collect the Right Logs the Right Way
Ensure that the SIEM system gathers, aggregates, and analyzes data in real-time from all relevant sources,
including applications, devices, servers, and users. While data is a vital component of SIEM capacity, you can
further support this by ensuring every facet of your organization is covered.
#3. Secure Endpoint Logs
An often encountered obstacle with endpoint logs lies in their continual changing, when systems are
intermittently disconnected from the network, such as when workstations are powered off or laptops are used
remotely. Moreover, the administrative burden of endpoint log collection adds a real degree of complexity. To
address this challenge, Windows Event Log Forwarding can be used to transmit a centralized system without
the need for installing an agent or additional features, as it is inherently available within the base Windows
operating system.
Stellar Cyber’s approach to endpoint logs supports a diverse range of endpoint logs, including Endpoint Detection and Response (EDR). By applying different alert pathways to certain subsets across different EDR products, it further becomes possible to accurately and precisely clean endpoint log information.
Stellar Cyber’s approach to endpoint logs supports a diverse range of endpoint logs, including Endpoint Detection and Response (EDR). By applying different alert pathways to certain subsets across different EDR products, it further becomes possible to accurately and precisely clean endpoint log information.
#4. Keep an Eye on PowerShell
PowerShell, now ubiquitous on every Windows instance from Windows 7 onward, has become a renowned tool for attackers. However, it is essential to note that PowerShell, by default, does not log any activities – this must be explicitly enabled.
One logging option is Module Logging, which provides detailed execution information about the pipeline, encompassing variable initialization and command invocations. In contrast, Script Block Logging monitors all PowerShell activities comprehensively, even when executed within scripts or blocks of code. Both of these need to be taken into account to produce accurate threat and behavior data.
One logging option is Module Logging, which provides detailed execution information about the pipeline, encompassing variable initialization and command invocations. In contrast, Script Block Logging monitors all PowerShell activities comprehensively, even when executed within scripts or blocks of code. Both of these need to be taken into account to produce accurate threat and behavior data.
#5. Take Advantage of Sysmon
Event IDs are vital to providing further context to every suspicious action. Microsoft Sysmon provides in-depth
event information such as process creation, network connection, and file hashes. When properly correlated,
this can help detect fileless malware that can otherwise evade anti-virus and firewalls.
#6. Alert and Respond
Despite the analytical power that machine learning grants to SIEM tools, it’s vital to contextualize it in the
wider scope of your overall security. Chief of this are your security analysts – an incident response plan provides explicit guidelines for every stakeholder, allowing fluid and effective teamwork.
The plan should appoint a senior leader as the primary authority responsible for incident handling. While this individual may delegate authority to others involved in the incident handling process, the policy must explicitly specify a particular position with primary responsibility for incident response.
From there, it comes down to the incident response teams. In the case of a large global company, there may be multiple, each dedicated to specific geographic areas and staffed with dedicated personnel. On the other hand, smaller organizations may opt for a single centralized team, utilizing members from various parts of the organization on a part-time basis. Some organizations might also decide to outsource certain or all aspects of their incident response efforts.
Keeping all teams cooperative are playbooks, which serve the foundation of mature incident responses. Despite the unique nature of each security incident, the majority tend to adhere to standard patterns of activity, making standardized responses highly beneficial. As this takes place, an incident response communication plan outlines how different groups communicate during an active incident – including when the authorities should be involved.
The plan should appoint a senior leader as the primary authority responsible for incident handling. While this individual may delegate authority to others involved in the incident handling process, the policy must explicitly specify a particular position with primary responsibility for incident response.
From there, it comes down to the incident response teams. In the case of a large global company, there may be multiple, each dedicated to specific geographic areas and staffed with dedicated personnel. On the other hand, smaller organizations may opt for a single centralized team, utilizing members from various parts of the organization on a part-time basis. Some organizations might also decide to outsource certain or all aspects of their incident response efforts.
Keeping all teams cooperative are playbooks, which serve the foundation of mature incident responses. Despite the unique nature of each security incident, the majority tend to adhere to standard patterns of activity, making standardized responses highly beneficial. As this takes place, an incident response communication plan outlines how different groups communicate during an active incident – including when the authorities should be involved.
5. Define and Refine Data Correlation Rules
A SIEM correlation rule serves as a directive for the system, indicating sequences of events that may suggest anomalies, potential security weaknesses, or a cyber attack. It triggers notifications to administrators when
specific conditions, such as the occurrence of events “x” and “y” or “x,” “y,” and “z” together, are met. Given
the vast amount of logs documenting seemingly mundane activities, a well-designed SIEM correlation rule is
crucial to sift through the noise and pinpoint sequences of events indicative of a potential cyberattack.
SIEM correlation rules, like any event monitoring algorithm, have the potential to produce false positives. Excessive false positives can waste the time and energy of security administrators, but achieving zero false positives in a properly functioning SIEM is impractical. Therefore, when configuring SIEM correlation rules, it is essential to strike a balance between minimizing false positive alerts and ensuring that no potential anomalies indicative of a cyber attack are overlooked. The goal is to optimize the rule settings to enhance accuracy in threat detection while avoiding unnecessary distractions caused by false positives.
SIEM correlation rules, like any event monitoring algorithm, have the potential to produce false positives. Excessive false positives can waste the time and energy of security administrators, but achieving zero false positives in a properly functioning SIEM is impractical. Therefore, when configuring SIEM correlation rules, it is essential to strike a balance between minimizing false positive alerts and ensuring that no potential anomalies indicative of a cyber attack are overlooked. The goal is to optimize the rule settings to enhance accuracy in threat detection while avoiding unnecessary distractions caused by false positives.
Next-gen SIEM and Log Management with Stellar Cyber
Stellar Cyber’s platform integrates Next-Gen SIEM as an inherent capability, offering a unified solution by consolidating multiple tools, including NDR, UEBA, Sandbox, TIP, and more, into a single platform. This integration streamlines operations into a cohesive and accessible dashboard, leading to a significant reduction in capital costs. Our SIEM log management is powered with automation that enables teams to stay ahead of threats, while the design of Next Gen SIEM empowers teams to effectively combat modern attacks. To learn more, you’re welcome to book a demo for our Next Gen SIEM Platform.