Skip to main content
Meet NetWitness at RSA Conference 2024!
Stop by our booth #254 or book a meeting with an expert. Reserve Your Spot Today!
The Language of Cybersecurity

Mastering the Art of Incident Response

  • by Ben Smith

Best practices and practical advice to protect your organization from external and internal threats.
A robust and effective incident response (IR) plan is no longer a luxury–it’s essential to a comprehensive cybersecurity strategy. From detecting early warning signs of a breach to ensuring swift and efficient recovery, a successful approach relies on proactive measures, well-defined processes, and continuous improvement.

Delving into the fundamental principles and best practices that drive a successful program can empower your business to stay one step ahead of the ever-evolving cyber threat landscape.

Incident Response Best Practices

From increasing awareness and preparedness to viewing time as a precious commodity, aligning organizational and technical plans, focusing on scoping, prioritizing documentation, and using the right tools to help, the following best practices can help your organization streamline and fine-tune your incident response process for the most success.

Increasing Awareness & Preparedness

Although IR plans are essential to managing security incidents effectively, they’re sometimes an afterthought. Ultimately, your organization can minimize the time required to remediate and reduce potential damages by ensuring awareness and preparedness.

Whether you call it a plan or a policy, your entire organization should be familiar with the IR motions before an incident occurs. That means conducting annual tabletop exercises involving managers and team members from each department. It’s not simply a technical exercise–beyond those teams responsible for diagnosing and remediating an incident, don’t forget to include your Legal, Compliance, Marketing, and any other teams involved. These exercises may last several days and typically include practicing various scenarios and comparing the organization’s response plan to the actual unfolding of a prior event. While less mature companies might seek assistance from external organizations with this type of expertise, more mature organizations might conduct these exercises independently.

Unfortunately, the importance of tabletop exercises is most apparent when organizations face disruptions they are unprepared for, increasing the time and the pain it takes to resolve the issue.

Viewing Time as a Precious Commodity

Time is crucial for diagnosing and remediating issues. Since people tend to struggle with performing well under extreme pressure, a well-documented IR plan can help guide their actions during an incident, which is helpful regardless of its size or impact.

The faster your organization can detect, contain, and remediate, the less damage a security incident can cause. Quick response times can prevent unauthorized access to sensitive information, minimize financial losses, and protect the organization’s reputation. It can also help reduce downtime and service disruptions, ensuring the continuity of critical business operations.

The longer it takes to respond, the more expensive and time-consuming the recovery process.

Aligning Organizational Incident Response with the SOC

Ideally, your organization’s overall IR plan should align with the Security Operations Center (SOC) playbook to ensure a cohesive, efficient, and effective response. A well-coordinated plan should define clear roles and responsibilities for team members, ensuring everyone understands their part in the process.

However, your organization’s overall approach might differ from the SOC’s in certain aspects.

Differences Between Organizational and SOC Incident Response Plans

While an organizational plan typically addresses the overall response strategies involving various departments and stakeholders, the SOC’s plan typically focuses on the technical aspects of detection, analysis, and response.

For instance, your SOC might rely on more detailed technical procedures, tools, and techniques specific to their activities and more detailed information on when and how to escalate incidents to relevant parties. In contrast, your organizational plan might have broader guidelines and procedures for the entire organization and outline reporting and escalation processes at a higher level.

Despite these differences, aligning the organization with the SOC can promote better management and reduce the overall impact of an event.

Focusing on Scoping

Scoping involves determining a security incident’s extent, impact, and nature. Understanding the scope can help your organization allocate resources and direct the right personnel, tools, and gear based on its severity and complexity.

In some cases, the scope could determine the reporting requirements to regulatory bodies or law enforcement agencies, so proper scoping ensures that your organization can meet its legal and regulatory obligations.

Finally, understanding the scope is essential for conducting a thorough post-incident review, identifying areas for improvement, and applying lessons learned to future response efforts. Post-mortems are a valuable tool that should never be bypassed “because we’re too busy now that the incident is over.” Your organizations should always include post-mortems as part of your IR plan.

Prioritizing Documentation

Proper contemporaneous documentation can ensure that information about the incident, response efforts, and decisions are communicated effectively among team members, stakeholders, and other relevant parties. Documenting the details–including the timeline, actions taken, and the findings–helps preserve critical evidence for later investigations and legal proceedings or to improve your organization’s future security posture.

However, when it comes to documentation and evidence collection–especially during a crisis–organizations should consider the following best practices.

Use Secure Communication Channels 
Some organizations rely on communication channels such as email for the document trail. However, email might not be the right place to store that documentation–especially while an event is ongoing, as the adversary might have access to corporate inboxes.

Be careful about how you communicate. Even communicating through a team’s existing collaboration channel could be compromising. Consider using a backup collaboration platform, or a different audio conferencing platform, for communication during an incident.

Keep Your Incident Response Plan and Documentation in Digital Form
Digitally documenting your organization’s IR plan has several advantages over physical documentation, including easy access, sharing, storage, real-time updates, searchability, and version control. Moreover, backing up a digital copy is easy, while physical documentation may be vulnerable to physical damage, loss, or theft.

Revisit Your Incident Response Plan Regularly
Regularly reviewing and updating your IR plan will ensure that it remains practical and relevant to your organization’s changing needs and the threat landscape. Thorough documentation of past incidents and response efforts can help you update the plan, incorporating lessons learned and addressing identified gaps. Exercise your IR plan at least once annually–it’s otherwise guaranteed to be out-of-date.

Maintain Backup Integrity
Documentation also plays a crucial role in ensuring the integrity of backups by providing information on when you made backups and what data you backed up. Any issues encountered during the backup process. Maintaining accurate and up-to-date documentation of backups can facilitate a swift recovery during data loss or system failure.

Coordinate with Disaster Recovery and Business Continuity Plans
IR documentation often complements Disaster Recovery (DR) and Business Continuity Plans (BCPs), providing valuable information on potential threats, vulnerabilities, and mitigation strategies. This information can help your organization develop more effective and holistic recovery and continuity strategies, ensuring resilience in security incidents and disasters.

Expanding Visibility Beyond a SIEM Solution

A traditional Security Information and Event Management (SIEM) solution can help your organization detect, investigate, and respond by providing a centralized platform for real-time monitoring, analysis, and management of security events.

SIEM systems aggregate and correlate logs from various sources, such as network devices, servers, applications, and security tools. This centralized log collection makes it easier for responders to identify patterns, trends, and anomalies that may indicate a threat. But having a SIEM in place is only the starting point, not the ending point, for comprehensive visibility across the environment.

Whether you believe that SIEM stands alone from a broader Extended Detection and Response (XDR) solution or is integrated within that XDR offering, your organization and the SOC cannot see everything it needs by relying on logs alone. Look for a wide-ranging XDR platform that provides native visibility into network traffic, endpoint activity, and logs. Ideally, XDR will also include analytics (User and Entity Behavior Analysis or UEBA) and workflows for the SOC (Security Orchestration, Automation, and Response or SOAR).

Fine-tuning alerts across all of these systems is also essential. Properly configured alerts ensure that security teams can identify, prioritize, and respond to potential threats while reducing the noise generated by false positives and irrelevant events.

Fine-Tuning Alerts

Too many false positives can cause security teams to waste time and resources investigating non-critical events, diverting attention away from real threats. Fine-tuning alerts can minimize false positives, allowing security teams to focus on genuine incidents and reduce alert fatigue.

Fine-tuning alerts is an ongoing process that involves regularly reviewing and adjusting alert rules and organizational requirements. More mature organizations may have a dedicated “content”-focused individual or team responsible for the maintenance and upkeep of the rules, scenarios, and threat intelligence that generate the alerts.

Incident Response FAQs

What are the first signs of a breach?

While incidents or breaches always feel unique from the perspective of the attacked organization, there are some common use cases that warrant careful attention, depending on the attack vector and the tactics, techniques, and procedures (TTPs) threat actors employ.

Here are some examples of potential first signs of an incident:

Phishing emails are one of the most popular methods for threat actors to gain access or a foothold within the target environment. In many cases, a phishing email will contain a link to a malicious or uncategorized domain that looks like a legitimate site, where the user might enter their login credentials, giving the threat actor access to resources held by the organization.

When users report phishing emails, organizations with network traffic-level visibility have an opportunity to analyze them and validate if there are any subsequent outbound connections. This Network Detection and Response (NDR) capability can also point to the compromised user’s device and all other compromised devices connecting to the same external malicious site. Automation can also help with phishing emails–tools that integrate directly with inboxes and analyze and scrape links to identify malicious indicators.

Newly registered domains may be legitimate or can be used for phishing campaigns or hosting malicious content. Although the age of a domain isn’t always something organizations look for, it can be a relatively simple way to identify potentially malicious sites.

A good rule of thumb is to pay special attention to domains registered less than 30 to 60 days ago. With some XDR platforms, you can create multiple buckets–for instance, domains that are less than 24 hours old, less than two weeks old, or in the 30- to 60-day range. Those different metadata points can be valuable indicators during an attack, and surfacing them directly to the responders as quickly as possible is paramount.

Insider threats are threats introduced by an insider, such as an employee or even a trusted contractor–but during an incident, it’s tough to determine if that insider is malicious or somebody who simply made a mistake. This is another key area where a holistic XDR offering can help answer this question through a UEBA capability, where the monitoring system learns and “knows” what typical behavior looks like from both users and entities (devices). Behavior outside an established baseline isn’t necessarily malicious, but these types of anomalies are important to record and investigate, especially within the broader context of all the other visibility data collected by the XDR platform.

What is the role of threat intelligence in incident response?

Threat intelligence provides timely, actionable, and relevant information about emerging threats, threat actors, and their TTPs. Whether organizations source that information internally or through an external organization, threat intelligence is the lifeblood running through the engines of the many tools designed to monitor and secure the environment.

However, because threats constantly evolve, threat intelligence can quickly become stale or outdated. Many organizations are focused on the ingestion and leveraging of threat intelligence but forget that, like all data, threat intelligence has a lifecycle, including a final or disposition phase where it can be (and should be) removed from production. Consuming threat intelligence in the absence of other tools and context is much less valuable, whereas having the ability to bring in that threat intelligence and fuse it directly into the context of an ongoing incident is essential.

It’s also critical to think about sharing selected intel with other organizations. There can be cultural and liability hurdles in taking this step, but in our experience, being able to share information with others in your same line of business or vertical always strengthens the community.

How to get started? It’s always easier to build trust between individuals than between organizations. Establishing a policy around information sharing and then sharing appropriate details with known and trusted contacts at other organizations–friends, past colleagues, etc.–may ultimately help you respond to an incident in less time.

How do you know when an incident is over?

Determining when an incident is over ultimately depends on the situation and understanding its scope. Collecting all the evidence and fully remediating the threat before declaring an incident “over” is critical.

Post-incident reviews (or post-mortems) are crucial in examining what worked and what didn’t, while lessons learned can help prevent future occurrences. The more effective the post-mortem, the easier it will be to identify gaps and fix issues.

Organizations should view security events as opportunities to become more innovative, enhance organizational protection, and reduce response time. Rather than focusing on blame during or after a breach, it’s essential to concentrate on moving forward, making improvements, and learning from mistakes. Even the most mature organizations make mistakes. Organizations can better understand what went wrong and implement the necessary changes by adopting a constructive approach after the incident response is completed.

Every Organization is Different

Each organization has unique requirements and different levels of expertise. But for every organization, it is critical to identify gaps in the visibility and scope by finding all relevant evidence. However, it can sometimes be challenging to determine how far back to look for evidence. Our advice is to follow the “breadcrumbs.” An XDR platform that stores network traffic, logs, and endpoint data for days, weeks, or even months provides plenty of those breadcrumbs which can ultimately diagnose and solve an incident.

Unfortunately, there is no “easy button.” Securing an organization is complex, always involving multiple vendors and platforms. Security architects also play a crucial role in understanding and resolving gaps using a combination of technologies and workflows–and solutions are guaranteed to change over time.

Mastering the art of incident response won’t happen overnight. As cyber threats become increasingly sophisticated, organizations must invest in robust IR strategies, training, and tools to stay ahead of the curve and maintain a strong defense against an increasingly unpredictable cyber landscape.

How NetWitness Can Help

With advanced threat detection, investigation, and IR capabilities, NetWitness helps organizations improve their security posture and respond to incidents more effectively by offering a range of features and functionalities.

  • Incident investigation provides robust investigation and forensic capabilities that help security analysts quickly triage and investigate security events with advanced query capabilities, visualization tools, and enriched metadata. 
  • Incident response orchestration helps streamline the IR processes with security orchestration, automation, and response (SOAR) capabilities, including features such as automated playbooks, case management, and collaboration tools. 
  • Threat intelligence integration for staying up-to-date with the latest threats by integrating with external and internal threat intelligence feeds for enhanced detection and response capabilities. 
  • Customization and scalability allow organizations to tailor the platform to their specific needs and requirements to remain practical and relevant as they grow and their security needs evolve. 

Learn more about how NetWitness can help your organization master the art of incident response, and schedule a demo today.