Practical Insights on Incident Response (with Brian Weidner & Chris Skinner)

In our latest State of Security podcast, Geoff Hancock, Global CISO at Access Point Consulting, delved into the complexities of incident response with two seasoned experts: Brian Weidner, CISO at A.O. Smith Corporation, and Chris Skinner, a former Marine Corps operations officer now leading the Incident Response (IR) team at Access Point Consulting.

Understanding the Incident Response Landscape

The discussion opened with Brian emphasizing the importance of building strong relationships with the executive suite, particularly the legal department, to ensure effective incident response. "Having a close workingrelationship with these teams is crucial," Brian noted, "especially when it comes to helping them understand the value of the cybersecurity measures you’re implementing." He elaborated on the necessity of communicating in the language of business leaders, ensuring they grasp how cybersecurity initiatives reduce risk and add value to the organization. Brian highlighted that when executives understand the impact of security efforts they are more likely to prioritize and support them, which is essential in a crisis. This alignment is critical when making critical decisions, such as involving law enforcement or allocating resources during an incident.

Chris added that while having an incident response process—no matter how ad-hoc—is beneficial, evolving that process into a structured and well-rehearsed plan is essential. "Your IR team should function like aspecial operations unit," he remarked, underscoring the need for a well-prepared and coordinated approach. Drawing from his military experience, Chris emphasized that the team's technical expertise is only part of theequation. The real challenge lies in ensuring everyone knows their role and that communication flows smoothly, especially during a crisis. He noted that inthe military, the success of an operation often hinged on meticulous planning and rehearsal, which allowed the team to execute effectively under pressure. Similarly, in incident response, preparation, and regular drills are crucial for ensuring that the team can respond swiftly and efficiently when an incident occurs.

Brian also touched on the importance of trust and credibility. He pointed out that when you have a solid relationship with your executive team, they arenmore likely to trust your judgment during an incident. "When they see youbcoming with that look, they know it's serious, and they’re ready to support you," he explained. This trust is built over time by not running to them with every minor issue but saving their involvement for significant incidents where their support is truly needed.

Chris echoed this sentiment, stressing the importance of having a clear and codified process. "An ad-hoc process is better than none, but without a structured plan, you end up with people pointing fingers and confusion," he warned. He added that the process should include clear guidelines on when tobrief executives, how to manage communications, and when to bring in externalpartners like law enforcement. By having these elements in place, organizations can avoid the chaos that often accompanies significant incidents and instead focus on resolving the issue.

Real-World Challenges in Incident Response

Both experts shared insights from their extensive experience, highlighting the complexities of managing large-scale incidents. Brian recounted a particularly challenging incident where threat actors had been active in a global organization’s network for over a year. "The threat actors were constantly shifting tactics, making it a significant challenge to contain and eliminate them," he explained. The attackers had gained deep access to the organization’s systems, compromising critical infrastructure like ActiveDirectory and effectively "owning" the entire environment. "We were dealing with an adversary who was not only entrenched but also adaptive. Every time we thought we had them cornered, they would pivot and find a new way to persist," Brian said.

The incident spanned multiple regions, requiring a coordinated global response. Brian emphasized the importance of timing and synchronization in such operations. "We had to execute containment measures simultaneously across different continents. Unfortunately, during our first attempt, one region jumped ahead, which tipped off the attackers and allowed them to adapt their tactics. It was a major setback," he recalled. This experience underscored the need for meticulous planning and flawless execution, as even a slight misstep could have significant consequences.

In contrast, Chris reflected on his military background, noting that while the technical aspects of incident response are often well understood, the real challenge lies in communication and process management. "Having a solid process in place is key," he emphasized, "especially when keeping executives informed without overwhelming them." He explained that in the military, managing a crisis involved executing the technical response and ensuring that the correct information reached the right people at the right time. "In a high-stress situation, the last thing you want is your executives hounding the IR team for updates every few minutes. It’s crucial to establish a communication rhythm that keeps them informed and allows the technical team to focus on the task at hand."

Chris also highlighted the importance of preparation and rehearsal in incident response. Drawing parallels to military operations, he pointed out that success often depended on the team’s ability to execute their plan under pressure. "We conducted crisis action drills regularly, which meant that everyone knew their role and what was expected of them when a real incident occurred. This kind of preparation is just as vital in the corporate world," he said. He warned that without regular drills and a well-practiced plan, even the best technical team could falter when an actual incident occurs.

Brian and Chris both agreed that while technology and tools are essential, the human element—communication, trust, and process management—ultimately determines the success of an incident response effort. Brian concluded, "At the end of the day, you can have the best tools and technology, but if your team isn’t prepared and your communication isn’t clear, you’re going to struggle when a major incident hits."

Leveraging External Support

The conversation then turned to the critical role of external partners in incident response. Brian shared a compelling example where collaboration with the FBI provided invaluable intelligence and added a layer of credibility when communicating with the executive team. "They brought insights that wewould never have had access to on our own," he said, emphasizing the strategic advantage of involving external agencies in critical incidents. The FBI’s involvement helped decode encrypted data, which was crucial to the investigation. Brian explained, "The threat actors had used a complex encryption method that stumped us initially. The FBI was familiar with this technique and was able to guide us towards the decryption, which was pivotal in our efforts to understand the full scope of the breach."

Moreover, involving a respected third party like the FBI can calm the executive team. "When the FBI is in the room, it underscores the severity of the situation and helps to validate the decisions you’re making," Brian added. This can be particularly useful when you need to convince executives totake actions that may disrupt business operations temporarily but are necessary to mitigate a severe threat.

Chris agreed, noting that cross-functional teams, which include third-party experts, can significantly enhance the efficiency and effectiveness of an incident response effort. "Bringing in external experts not only provides additional technical expertise but also introduces different perspectives that can be crucial in navigating complex incidents," he said. Chris pointed out that in the military, joint operations often involved working with allied forces and other agencies, which taught him the value of collaborative efforts. "In incident response, it’s much the same. No single entity has all the answers, but together, you can piece a more complete picture and respond more effectively."

Navigating New Regulatory Requirements

Most of the discussion focused on the upcoming Cyber Incident Reporting Act, which will require organizations to report incidents within 72 hours. Both experts expressed concerns about the potential for increased regulatory burdens but acknowledged the importance of timely reporting in managing cybersecurity risks. Brian questioned how these new regulations would integrate with existing requirements, emphasizing the need for clarity to avoid overwhelming organizations with redundant reporting obligations. "We already have to report to multiple regulatory bodies depending on the industry and the nature of the incident," he pointed out. "Adding another layer could create confusion and potentially slow down the response process if how these regulations interact is not clearly defined."

Brian also highlighted the practical challenges of meeting the 72-hour reporting requirement. "In a complex incident, it’s not always possible tofully understand the scope and impact within 72 hours," he explained. "The concern is that organizations might be forced to report incomplete or inaccurate information, which could lead to further complications down the line."

Chris added that while reporting is crucial, it must be balanced with the practical realities of incident management. "The last thing you want during a major incident is to divert resources away from containment and remediation to meet a regulatory deadline," he cautioned. Chris emphasized the importance of having a streamlined process for gathering and reporting information, which can help meet regulatory requirements without compromising the effectiveness of the incident response. "Preparation is key here as well," he noted. "Having templates and predefined communication channels can make the reporting process more efficient, allowing the IR team tofocus on resolving the incident."

Both experts agreed that while the intent behind the Cyber Incident Reporting Act is understandable, its implementation needs to be carefully considered to ensure it supports, rather than hinders, effective incident response. "It’s about finding the right balance between regulatory compliance and operational effectiveness," Brian concluded. "If done correctly, it can enhance transparency and trust, but if mishandled, it could become just another bureaucratic hurdle."

Key Takeaways: Preparation is Paramount

The webinar concluded with a resounding message: preparation is the cornerstone of effective incident response. Brian and Chris emphasized the critical importance of having a well-developed, well-rehearsed incident response plan that integrates internal and external resources. "You have to be ready at all times—because when an incident happens, there’s no time to start planning," Brian succinctly stated, underscoring the urgency of being proactive rather than reactive.

Chris reinforced this sentiment by highlighting that preparation isn't just about having a plan on paper; it's about ensuring that every team member knows their role and that communication channels are precise and efficient. "Regular drills, strong relationships with external partners, and a clear understanding of regulatory requirements all play a crucial role in ensuring your team is prepared when it matters most," he added.