Threat led penetration testing – the past, present and future

Stewart K. Bertram | 11 July 2017

Threat led penetration testing is, in essence, using threat intelligence to emulate the tactics, techniques and procedures (TTPs) of an adversary against a real time mission critical system. The concept is currently being implemented in a number of ‘flavors’ around the globe including schemes such as the UK’s STAR (Simulated Target Attack and Response) or CBEST scheme, the Netherlands TIBER (Threat Intelligence Based Ethical Red teaming) scheme and the Hong Kong based iCast schemes.   

The recent quarter has seen some extremely significant development within the realm of threat led penetration testing (TLPT). The concept of TLPT is rapidly expanding beyond the United Kingdom. TLPT advances the boundaries of conventional penetration testing by seeking to adopt the tactics, techniques and procedures of an advanced threat actor aggressively targeting a critical system. You can read more about TLPT in a previous blog. Our work with the first two Dutch TIBER projects, as well as our workshop at the Bahrain International Cyber Security Forum & Expo, are great examples of this.

Given this expansion, I wanted to review of where the TLPT concept has come from and where it may be going to.

The Past

The origins of TLPT began with the UK’s CBEST scheme in 2013, to which Digital Shadows was a major contributor both in terms of the development of the original framework and the implementation of the actual projects.  Since then, there have been around fifty CBEST style engagements of which Digital Shadows has carried out the majority, from which three lessons have emerged:

  1. It has to be testable. Any testing scenario put forward by the Threat Intelligence provider has to be testable by the penetration test partner, with the scope of their capabilities. In practice this means less abseiling through open windows and more focus on technical exploits such as the indicators of compromise associated with specific threat groups.
  2. The importance of the ‘golden thread’. This is an easy concept to outline but a challenge to implement. As the report moves from the initial quantitative, data collection stages to the later, qualitative scenario building, the report should create activities linking data, information and intelligence into a “golden thread”. In practice doing this is really quite simple, for example taking client emails that have been implicated in various data breaches and focusing phishing campaigns against them.
  3. Creative versus effective scenarios. The culmination of a TLPT (the TI phase at least) all revolves around the attack scenario following the Exposition, Rising Action, Climax, Falling Action and Dénouement structure. While it can be tempting to devise elaborate scenarios, its important to remember that the core objective of a scenario is to successfully compromise the target system at the lowest level technically possible.

The Present

The CBEST scheme has been a huge success, which has led to the concept of TLPT being expanded beyond the financial services in the UK. Currently Hong Kong and the Netherlands have ‘in flight’ schemes with Singapore and the United States considering implementing their own proprietary schemes.

  • Sector diversification is happening, specifically across the telecoms, nuclear, wider energy and even space sectors. Although the sectors are varied, the principle of TLPT is the same – to test real time in flight critical systems using the TTPs of real world threat actors.
  • Regional expansion is rapidly occurring with Hong Kong, Singapore, The Netherlands and the USA all looking to develop and implement variants of TLPT.

The Future

it is worth a speculating about some of the features that I feel will become fixtures within TLPT in the future.

  • Iterative development of scenarios within the penetration test phase. Future TLPT will iteratively update the threat profiles based upon the results of the penetration test phase. This will result in a set of scenarios that are all viable but only under specific sets of circumstances. This would shift the current scenarios metric from ‘viable or not?’ to a more nuanced ‘viable under these circumstances. This could be a high-level insider threat with zero day vulnerability. This would create a situation where defenders could then assess the likelihood of a scenario coming to fruition based on the threat actor’s and level of defences.
  • Reuse of the result. On average, the results of a TLPT have a shelf life of between 18 and 24 months. Therefore, the organization has the opportunity to reuse the final result for a number of technical and non-technical exercises, such as a crisis management workshop for executive leadership.
  • Broader range of organizational testing. There is huge potential of TLPT to expand out from just being a technical test to encompass non-technical element of the client organization risk management framework, such as crisis management workshops and media management workshops.

The success of the CBEST scheme and the subsequent expansion suggests that threat led penetration testing is an exciting trend. Of course, CBEST and TIBER are evolving rapidly, and so predicting the future adoption by providers and users is unknown. However, by building on past successes and learning lessons, threat led penetration testing could go from strength to strength.