The security guard, patrolling a corporate campus, spots a broken window and records their assessment. Following a predefined checklist, the guard finds a footprint in the broken glass inside the building, which warrants escalation: the guard to their superiors to detectives to the organization’s leadership. Following the trail of evidence, the detectives reach a locked safe. Did the safe work and deter the thief? Was the thief able to bypass the safe and get what they wanted? Is the corporation able to enumerate everything that was in the safe?
Regardless of the outcome, the story resonates with a reasonable approach to physical security and response given that we have dealt with this type of scenario for a very long time.
Challenge for Incident Response Companies in Cybersecurity
So why is this so difficult within the cyber domain? Many companies overcomplicate their incident response programs when the approach should be as direct as the story above.
Stepping out of the fire of reactive response to think systematically about the structure and rigor of an operational program is when you can define an approach that is more efficient right of breach (after) and more effective left of breach (before). To get you started, let’s:
- review Jargon
- explore the Principles of Threat Detection and Incident Response
- define the Roles of an Incident Response Team
- review Why Hunting Matters for rapid incident response
- classify the Content required for threat detection and incident response services for businesses
- present Next Steps for incident response companies
Understanding Incident Response Terminology
Before proceeding with program building, it is important to have a foundational understanding of common terms.
- CSIRT – Computer Security Incident Response Team that handles any security incident that bypasses security controls.
- SOC – Security Operation Center handles administration and monitoring of security controls (firewalls, IPS, endpoint protection, DLP, etc.).
- Event – Anything that happens within the computing environment (e.g., log records, network sessions, file changes, permission changes, privilege escalation, command execution, memory changes). Requires collection.
- Incident – An event or collection of events with indicators that align with your threat priorities. A physical example would be a broken window. Requires investigation leading to remediation or escalation.
- Major Incident – An incident with the potential for loss or harm. Requires leadership notification.
- Breach – An incident with evidence of loss or harm. Requires activation of the Executive Response plan with public and/or customer notification.
1. Business Defines Risk in Incident Response Programs
Task: Create Risk Register with Threats and Critical Assets
This critical step shapes the rest of your rapid incident response plan. This is where you understand what assets need to be protected and why, as well as the basic information needed to understand mitigating threats to those assets.
2. Threat Intelligence Defines Controls and Priorities for Rapid Incident Response
Task: Align Controls to Mitigate Controllable Threats
Task: Cultivate Threat Intelligence for remaining Threat Priorities
When presented with 10 active threats and you have capacity for two, where do you start? Threat prioritization is critical to your success. If you can implement a security control to mitigate your threat priorities, do it. Things like Multi-Factor Authentication (MFA), Risk-Based Authentication, Network Segmentation, IPS, Firewalls, and so on are all great ways to reduce the number of threats you must hunt for.
Next, determine how best to get information on remaining threat priorities not mitigated by controls. Identify open-source feeds, paid subscriptions, and information sharing – anything that will help you be aware of how these threats behave, when they are active, and how they might show up in your environment.
3. Establish an Incident Response Team Plan around Threat Priorities
Task: Develop Use Cases for your Threat Priorities
Conduct threat analysis on the information gathered in the first two steps to define detailed use cases. These use cases identify the nature of the threat, stakeholders, the critical assets likely to be affected, the types of content needed to detect an active threat, as well as the associated logic to detect it, and guidance on prioritization and workflow.
With this level of understanding, you can better structure the team and develop communications and incident management workflows, enabling you to build an incident response team plan to manage the chaos.
4. Operationalize Incident Handling with Incident Response Companies
Task: Combine your Use Cases into Playbooks
Use the incident response team. Plan to establish response procedures and operationalize use cases into actionable playbooks. Primary incident handlers will execute against them daily. This addresses all threat priorities detected through structured logic, correlation, and investigation.
5. Hunt Daily: An Essential Phase of Threat Detection and Incident Response
Task: Hunt for Anomalies that exist outside your playbooks
Any threat priorities not covered by playbooks will require hunting. This is where the most critical rapid incident response activity occurs. This is the stage where zero-day exploits are found and anomalous activity that bypasses both defense-in-depth and operational rigor occurs. Hunting is continuous and constantly informed by emergent threats and new developments within the threat landscape.
6. Commit to Continuous Improvement in Incident Response Programs
Task: Review incidents quarterly and critical incidents directly
Task: Exercise playbooks through Simulation/TTX for readiness
Task: Assess resilience to threats with Gap Analysis
Once the program is established (steps 1-5), it is time to reinforce skills, orient new staff and train for any new use cases or threats identified. Ongoing use of tools, such as gap analysis, helps track overall progress and identifies additional improvements needed.
Defining the Roles of an Incident Response Team
With the principles of threat detection outlined, it is time to turn to the core roles of a successful incident response team. Understanding how these functions perform is critical to assessing where you are on your security journey.
- Threat team focuses on managing threat intelligence. They identify the data feeds required for information on threats and which records within those feeds are valid.
- Content team leverages threat intelligence reports to refine the use cases necessary to detect the presence of threat priorities and build the associated playbooks. This includes defining rules, building parsers, configuring alerts, and documenting response procedures, as well as tuning the detection strategy to minimize false positives.
- Playbooks (triage) team executes against the standard procedures, validating any alerts, and escalating to the hunting team if the playbook does not result in a path to remediation. Most incidents are handled at this level (in a large organization, 60-90 incidents per day are not uncommon).
- Hunting team spends most of the time on proactive investigations, looking for anomalies or following new threat intelligence as shared by the threat team. Hunters are also responsible for investigating incidents that cannot be resolved via playbook. Major incidents are handled at this level and should be rare.
These roles are necessarily separate from current security device and network administration teams – a key distinction for all incident response companies. The device and network administration team spends its time maintaining firewalls, core networks, AV systems, or servers. As such, they also cannot investigate incidents, manage threat intelligence, tune content, or hunt.
These roles may represent a fraction of a full-time employee or multiple team members, but the functions are critical to a comprehensive IR strategy. In smaller teams, cross-training team members to avoid single points of failure enables consolidation.
Managed Detection and Response: Extending Incident Response Services for Businesses
You can outsource the playbooks’ role in providing 24×7 incident triage by partnering with a managed detection and response provider (MDR) if you have well-defined playbooks. MDRs deliver rapid incident response by executing playbooks rather than the traditional log aggregation and alert hand-off provided by a traditional Managed Security Services Provider (MSSP), thus alleviating the burden off the in-house team for round-the-clock staffing. The in-house team’s focus is then on threat intel, content analytics, and hunting.
Unfortunately, hunting can never be fully outsourced. Third-party providers simply cannot understand the environment as well as the in-house team, thus requiring at least one person in-house capable of handling playbook escalations and anomaly detection (the hunter described above). They should spend most of their time looking for suspicious activity within the environment.
Retainer and Surge Response Capabilities for Incident Response Companies
It may prove useful to further round out capabilities with an incident response retainer. This is separate from an MDR provider who typically does not provide advanced forensic capabilities.
The expert resources available on retainer enable you to add resources to deal with a major incident while ensuring your existing team can continue with their daily operational responsibilities.
A common threat actor tactic is to create a noisy event to mask a far subtler one. The in-house analysts need to continue with daily tasks while the expert resources investigate as quickly and efficiently as possible.
Ultimately, understanding how these incident response team roles map to the staffing model is critical to developing a sustainable operations plan.
Remediation Responsibilities: Separating IT and the Incident Response Team
NIST 800-61r2 outlines the major phases of the incident handling process as Preparation, Detection and Analysis, Containment, Eradication, Recovery, and Post-Incident Activity. However, while the IR team defines the remediation strategy during the Containment phase, it is the IT organization that executes the remediation strategy during the Eradication and Recovery phases.
While remediation is technically part of the NIST guideline on incident handling, incident response teams should not conduct remediation. An operational incident response team must focus on playbook execution, hunting, threat intelligence, and content analysis, not on IT operations.
One of the biggest reasons organizations fail to implement a successful incident response program is because they approach it as an additional duty for their IT or IT Security teams when it is necessarily a separate operational function.
Why Hunting Matters in Threat Detection and Incident Response
An active threat attempts to exploit a critical asset. Does your defense-in-depth prevent it? Ideally, defense-in-depth prevents the attack using a combination of signature-based solutions, architecture, and blocking – a win for your security operations.
If the attack successfully bypasses your defense-in-depth, do you have a playbook in place? A playbook enables the incident response team to detect and respond to the attack – a win for incident response.
However, determined attackers can get past your playbook procedures and progress towards your critical asset. The hunting team should detect the anomalies created by this activity and disrupt the attacker’s dwell time before they complete their actions on objectives.
Hunting is the last line of defense against the most determined attackers, those most likely to cause the greatest amount of damage.
Operational Content Needs for Threat Detection and Incident Response Services for Businesses
Content is needed to build hunting and detection capabilities and is where many organizations struggle most.
Why is that? Most often it is a result of competing priorities. Multiple teams within an organization often need the same data but for different, competing business cases, such as compliance, operations, detection and response.
- Compliance: Demonstrates an organization doing what they said they would. This information primarily comes from system and application logs and is presented via reports.
- Security operations: Seeks to aggregate security alerts, monitor security devices, and respond to known bad events with a standard procedure. This information primarily comes from security device logs, sometimes supplemented by NetFlow, and is presented via alerts and dashboards.
- Detection and Response: Generally, deals with all the information outside of Compliance or Operations. Threat hunters interface with an event database to look for anomalies based on context logs, NetFlow, packets, and endpoint forensic data.
Operational incident response is most effective and timely when built on three core elements of content intelligence:
- Context from Logs: Key log data provides visibility into activity on critical systems that inform and frame who, what, where, and when.
- Evidence from Network: Packets contain the truth of how a threat was executed and patterns of activity associated with specific exploits and command and control.
- Proof from Endpoints: Endpoint forensic data is vital for understanding how a threat succeeded.
Combining all three increases the analyst’s speed and efficacy by an order of magnitude. If only choosing two, Packets and Endpoints are the most crucial, as they still provide the most visibility into an attacker’s tools, tactics, and procedures.
Next Steps for Building an Effective Incident Response Team
This is just the beginning but provides a solid foundation from which to build your incident response program. Logical next steps include:
- Complete a Gap Analysis and Roadmap
- Build a Threat Intelligence Program Roadmap
- Formalize the Incident Response Plan
- Develop Tactical Playbooks
- Establish an Incident Response Retainer
- Conduct a Controlled Attack and Response Exercise
- Conduct Tabletop Exercises
With the right guidance, you can build a successful operational incident response program in twelve to eighteen months. Focus on quick wins and build incrementally. Target your critical assets, determine the most likely threats to those assets, and then build playbooks around identifying those threats. Once you start executing against these playbooks, expand to new assets and new threats.
Every organization is unique, and it is up to you to find the right combination of internal capabilities that leverage external MSSP, MDR, Retainer, and advanced cyber defense services that make the most sense.
Visit us to learn more about building an incident response operation and explore how incident response companies deliver rapid incident response, comprehensive threat detection and incident response, and effective threat detection and incident response services for businesses.