Until the creation of the European Union’s Digital Operational Resilience Act (DORA) (2022/2554), there hasn’t been much need for those of us in the U.S. to be aware of Threat-Led Penetration Testing (TLPT). DORA’s text also isn’t explicitly clear around what this TLPT is, or what makes it distinctly different from the average penetration testing. This is very similar to other DORA requirements (such as the ICT Risk Management Framework) where most of the useful details of the standard are ambiguously mentioned. However, if you’re just now starting to be exposed to DORA, the secret sauce is in the associated Regulatory Technical Standards (RTS), such as the DORA RTS for ICT Risk Management. With that background, here’s a quick summary of what I expect DORA’s TLPT requirements to be, based on their underlying standard, which is the TIBER-EU standard. If you’re curious, I’ve also included links at the end to the official documentation.
TIBER-EU was standardized in 2018 and is an acronym for Threat Intelligence-Based Ethical Red Teaming as the de-facto European Central Bank standard. It also serves as the basis for DORA’s requirements around Threat-Led Penetration Testing (TLPT). Unlike traditional penetration testing, intelligence-led red team tests provide a minimum of three scenarios for targeting across the entity’s entire surface; including people, processes, and technologies. With a testing interval of three years, these tests utilize recognized external resources, though there is limited provision for “internal” testing. These internal tests are limited, such as allowing a red team on a subset of testing, under limited circumstances, and under DORA supervisory approval.
Scoping (as always) is critical, but must include the Critical Functions, which are defined as:
Critical Functions are the people, processes, and techniques required by the entity to deliver a core service, which, if disrupted, could have a detrimental impact on financial stability, the entities safety and soundness, the entity’s customer base, or the entity’s market conduct.
Not only must testing include these Critical Functions, but unlike previous scoping exercises you might have encountered, this scope must be attested to by the Board of the entity, as well as being validated by the EU authority. Each entity is responsible to perform the planning, project management, and testing according to the requirements. This however does have the benefit of the entity being in control of the testing process.
During testing, a secured White Team is formed inside the entity, using only trusted contacts at the top of security escalation chains to avoid miscommunication and data leakage. This team also has the ability to halt testing when they suspect that tests might have a damaging effect against production infrastructure. All knowledge of testing is restricted, as any knowledge that modifies standard practices will lead to invalidation of the test. The entity’s Blue Team must never be aware of any testing activities until they have been completed. A RACI showing these relationships is available in the below resources.
Also unlike traditional penetration testing, TLPT reaches a rigor in access and planning not traditionally experienced by many teams. Because of this, I suspect that many teams will experience findings and failures not previously identified. This rigor comes from extended engagement times (4-6 weeks) for planning and preparation, as well as a minimum requirement for twelve (12) weeks of active testing. The testing vendor also has the ability to gain “leg-up” scenarios, such as pre-provisioned network access to emulate a persistent attacker who has to invest significant time in preliminary attack vectors before engaging Critical Functions, without the need to extend their testing time beyond the required twelve weeks.
The activity of Threat-Led Penetration Testing (TLPT) requires two vendors, both a Threat Intelligence (TI) Provider, as well as a recognized Red Team Provider. These vendors will work together to generate and utilize current threat intelligence in an attempt to successfully compromise one or more Critical Functions and their underlying production systems. Measurement of this success is established using predefined flags to “capture”.
Because this testing is intelligence-led, it requires a recognized provider to drive the testing strategy via a bespoke Targeted Threat Intelligence Report (TTIR) the identifies key targets, along with their associated threats based on likely APT activities.
The TTIR contains this information in the form of TTPs, and is customized for the entity’s situation (e.g., sector, geographic location, etc.) The associated examples for testing exceed those found in historical penetration testing, and include various forms of social engineering, or even physical location compromises to attempt successful compromise of the Critical Functions. The Threat Intelligence Provider (TI) will seek to gather and utilize information about the entity in generating their report. This includes business and technical overviews for each Critical Function, which will be combined with their own intelligence to create the report/recommendations. The entity being tested can provide additional details around the Critical Functions as they find desirable. The resulting, final report will provide tailored attack scenarios, anticipated threat actor goals, as well as motivations, and be summarized with validated evidence to underpin the business case for remediation.
After this preparatory step has been completed the Red Team will begin testing. This testing is highly confidential, and is performed against production systems, using code names, and without the entity’s security team’s knowledge. The only exception for security team knowledge is the already-discussed White Team members. Because of the nature of live security exploitation against production systems, a readiness check should be accomplished beforehand to ensure that backup and restore (DR) capabilities are in place and functioning as intended. Risk management has a defined process in Article 5 of the Final Report on Regulatory Technical Standards on TLPT, but has an emphasis around ensuring the quality of Red Team members associated with the activities. It also includes guidance in other areas, as the risk of unintended consequences from Red Team activities, regardless of level of care, does have the possibility of business impact during testing. It is noteworthy, that under “exceptional circumstances”, and when all alternative options have been exhausted, there is the ability for limited purple teaming. This is only applicable to validate specific security measures, tools, or techniques in a controlled environment.
After the conclusion of the Red Team engagement and testing, a report will be provided. The Blue Team (previously unaware of the ongoing activities) is also required to provide a report of their corresponding actions during this time period. Both reports are then used to create a replay workshop, which is referenced in DORA as a Purple Team. This leads to a review session with the EU authorities, including the creation of a Test Summary, as well as a Remediation Plan. Once reports and a remediation plan have been created, a formal attestation from the entity, both vendors, as well as the lead authority must be secured around the validity of the testing process.
There’s my two-cents worth of summarizing the Tiber-EU framework and process which underlies the DORA regulations facing U.S. businesses who are now in scope of this newer regulation.
Because of the extended testing schedule, required vigor and attestation on scope, the ability for the Red Team to have “leg-up scenarios”, as well as the focus on APT-level behaviors; I do suspect that many entities will experience compromises, even if current penetration tests are returning significant findings. My personal opinion is that these tests aren’t about who isn’t compromised, but instead about how difficult you make compromise achievable, and how your Blue Team (hopefully successfully) detects and defends against the Red Team.