Why red team exercises should be included in your company’s supply chain defense strategy

Software supply chain attacks are an increasingly concerning threat responsible for breaches that annually cost billions of dollars in operational damages to companies and impact the safety of end users. 

One of the more recent examples of these attacks is the 3CX breach that targeted customers of the voice-and-video conferencing application used by thousands worldwide.

In a white paper produced by Juniper Research, it’s been estimated that cyber attacks targeting software supply chains will cost the world economy $80.6 billion in lost revenue and damages annually by 2026.  

In this article, we ask if there is a feasible strategy for organizations that operate software supply chains to reduce the astronomical damage costs projected by 2026?

It is an ongoing challenge that the cyber industry combats today and ultimately affects everyone.

What is a supply chain attack?

A supply chain attack refers to a cyber attack that targets the software or hardware supply chain of an organization or system.

Instead of directly attacking the target organization’s network or IT infrastructure, the attacker focuses on compromising a trusted supplier or vendor that provides components, software, or services used in the target’s supply chain.

The goal of a supply chain attack is to exploit the trust relationship between the target organization and its suppliers. By compromising a supplier or injecting malicious code or hardware into the supply chain, the attacker can gain unauthorized access, exfiltrate data, introduce malware, or disrupt the operations of the target organization.

This approach allows the attacker to bypass traditional security measures since the compromised code or hardware may be perceived as legitimate and trusted.

Supply chain attacks can have severe consequences. They can affect a wide range of organizations and industries, including government agencies, financial institutions, healthcare providers, and technology companies.

A ubiquitous challenge

The software supply chain is everywhere and most modern humans interact with it daily. It’s crucial for everyone, directly or indirectly, such as when our devices update to the latest firmware version or when a developer pushes code into a company’s project code repository to update and build a new version of an application, potentially with thousands of supported end users.

Most of us leverage and benefit from the services software products provide, which means we depend on the security of a vast upstream of proprietary, third-party, or even open-source software libraries, code, and configurations.

Organizations that operate, contribute to, and own these vital software supply chain systems are responsible for defending and maintaining the integrity of these pillars of source code resources for the safety of those that depend on them.

The security of modern businesses, governments, and individuals further down the supply chain could be impacted if they were to fall under the control or influence of threat actors. This is, again, a massive challenge for cyber as an industry today. 

To face these emerging public safety challenges and rising security risks in software supply chains, a comprehensive strategy dedicated to proactive defense is necessary to ensure the security and resilience of these crucial systems.

Why are attackers still largely successful, leaving chaos and limited attributable post-attack information?

Some of the current limitations in the industry and why these attacks are still and will be an ongoing threat can be summed up as follows:

Bias gaps

Considerable bias gaps exist between how organizations expect a threat actor to carry out a software supply-chain attack against their organization and how they are genuinely attacked.

Organizations across industries must clearly understand how attackers can breach their software supply chain systems. But at the same time, gaining better situational awareness around the issue across industries when applied to an organization’s specific network environment is not easy.

This becomes more evident when the problem is analyzed from a macro perspective. There are too many components, diverse technology stacks, and ways to architect and configure software supply chain systems.

Exploring several breach case studies highlights this issue. Listed below are some examples. Each breach is different, and attackers get notably more innovative and dangerous with each successful violation, continuously adapting their techniques to a target network.

• SolarWinds – sunburst malware strains leveraged trusted Orion network traffic (OIP) to hide activities.
• British Airways – threat actors leveraged trust in third-party contractor networks to gain initial access.
• Microsoft Azure – Pipelines vulnerability – Showed us that trust in top-tier cloud services can also be leveraged and abused for supply chain attacks.
• 3CX – More recently, exploited multilevel implicit trust in the integrity of upstream application software.

Real time identification

Gaps in identifying the incident and establishing effective collaborative communication channels toward an effective and decisive response as the attacks occur in real time.

Suppose you don’t know how these supply chain attacks can be carried out and you’ve never had a cyber offensive red team seek to breach the organization’s software supply chain asset. In this case it will be tough to understand how to best identify the attack as it happens in real time, especially with the element of surprise being on the attackers’ side.

Network defenders or your blue team still need to parse and correlate copious amounts of raw log data from monitoring tools to get situational awareness of a network. On the other hand, attackers know precisely where they are, always, what they are doing, and how.

While defenders collect and parse buckets of information from many different sources, attackers hide their activity in assumptions and ‘trusted’ spots within the network. 

These limiting gaps are why such notably wide-scale breaches go undetected for long periods of time without raising significant incident response alarms.

Supply chain attack impact

Overall, due to insufficient awareness of how the attack manifests across a wide array of industries that maintain and deliver packaged code down a supply chain system to end users, this is likely to translate into a damage cost that is already relatively high.

This with a total cost this current year of $46 billion according to Juniper Research—all in the hands of often well-prepared state-sponsored adversaries compromising insecure but critically vital software supply chains. 

The impact of these attacks is tremendous. A complete compromise of end users’ privacy, safety, and integrity is at risk. Regarding supply-chain attacks, a general strategic shift is necessary to defeat the industry’s prevalent security bias gaps and, at the same time, mitigate adversaries’ attempts to compromise vulnerable supply chains.

Best practices

Of course, checking each of the usual internal best practice security control boxes helps and is highly recommended to strengthen a supply chain defense strategy: 

• Network segmentation
• Mature vulnerability management process
• A well-defined, matured, secure software development lifecycle process.
• Strong password policy
• Access control systems
• Principle of least privilege
• Pentest activities
• Multi-Factor Authentication (MFA), FIDO and WebAuthn
• Ensure monitoring and logging on critical systems and mapping of organization risks
• Understand the organization’s threat model and threat actor capabilities per their resources (MITRE, EDR)
• Enforcement of internal and external information security/law policies

However, with the ever-evolving and adaptive state-sponsored threat challenges that the cybersecurity industry faces today, an ongoing race to infinity between offense and defense exists.

What can organizations do?

Organizations must find a way to avoid dismissing this highly competitive security balance race with a business-as-usual, time-limited (often with window time), vulgaris pentest scope.

The risks and impacts are too significant and have a long reach for organizations and end-user safety. 

Suppose an organization is mature enough to maintain a successful software supply chain with a tested, secure software development lifecycle as a pillar. In that case, said organization should also be mature enough in the same way to undergo red team engagements for the sake of its software supply-chain security and the safety of its end-users. 

The standard best practice industry tools deployed internally and listed above are necessary and often required by internal policies or law. But in today’s adaptive threat landscape, more is needed. 

Attackers understand how defensive controls work and behave.

Within their malware development process, they can simulate and test their malware’s capabilities against known security solutions before deploying.

While network defenders hope that security controls will catch novel attacks with the latest product updates, adversaries are armed with the knowledge of how defensive tools will react against their malware.

This is a significant area for improvement in current supply chain defense strategies because companies across industries still expect and rely on tooling to detect new, never seen attacks. 

Still, in principle, security controls don’t adapt fast enough. Precisely because attackers know how available security controls work and react before the malware is deployed. 

Adaptive threats also invest heavily in security research and skill areas to develop new and unexpected ways to infiltrate and bypass technical security tools. By the way, this assumes that such security tools have been configured correctly in the first place, which, if not, is also a security liability/risk for organizations.

What is needed to help mitigate modern supply chain threats?

An additional, collaborative, offensive security layer that consistently challenges an organization’s security bias and supply-chain defense procedures through data-led ‘trial by fire’ is necessary.

Companies should widely adopt objective-based red team exercises against their critical high-value supply-chain-related assets. It is ultimately one of the best strategies to progressively and consistently identify hidden security gaps and challenge risky internal security status-quo assumptions. 

With access to an appropriate red team capability program and having it collaboratively integrated into the supply-chain defense strategy, organizations can: 

• Reduce security biases
• Consistently, progressively, and adaptively strengthen the organization’s strategic approach to defending its software supply chains
• Identify and close breach opportunities before such catastrophic incidents happen. Potentially saving millions of dollars in downstream damages and fines
• With an engaged Security Operations Center (SOC) in each red team engagement, an organization has opportunities to exercise and improve existing Digital Forensics and Incident Response (DFIR) capabilities  
• Have better situational awareness regarding their specific supply chain systems, staff, and procedures

Why proactive red teaming can be effective in identifying supply-chain breaches before they happen

Let’s look at a few examples of previous supply-chain breaches.

At a high level, a clear pattern can be dissected of threat actors exploiting unexpected trust relationships to gain initial access, evade detection, and achieve action on objectives with often novel, hard-to-detect, and highly evasive living-off-the-land lateral movement techniques throughout the network.

Adversaries reach the organization’s exposed software supply chain to deploy malware code developed and tested to deliver and evade that day’s EDR capabilities.

SolarWinds 

Perhaps the most critical example is the SolarWinds 2019-2020 hacks, where the Sunburst malware strain deployment went undetected for 14 months.

We see that attackers within their malware development capabilities mimicked expected network traffic, scheduled actions, and implemented capabilities that were notably not picked up by tools at the time. 

Only a couple of tools automatically picked up on the deployed threats. But even so, the attackers were primarily successful in bypassing most security solutions at the time because they tested their malware against known security controls.

They also knew precisely how Endpoint Detection and Responses (EDRs) would behave before deploying into the compromised supply chain as a signed trusted “update” Dynamic Link Library (DLL), which ended up being a historical and significant safety threat even to top-tier government agencies and cybersecurity firms.

The only reason the malware was caught long after its deployment is that a downstream FireEye employee noticed an unexpected MFA device enrollment notification during a critical Biden vs. Trump election campaign. 

The SolarWinds hacks have since catalyzed a fast and broad change in cyber. Many companies and government agencies are still in the process of coming up with better ways and methodologies to defend and react to supply-chain attacks. 

The lengths to which SolarWinds hacks went to evade detection were quite impressive:

• Digitally signed, valid, malicious DLL files used by attackers. This meant that a compromise of software development and distribution pipelines took place, resulting in successful and undetected infiltration to the software supply chain itself
• Covert malware network channels using the trusted Orion network protocol (OIP)
• Low malware source code footprint, considering the trusted Orion development toolchain was leveraged to deliver the sophisticated malware

In the words of FireEye’s CEO after disclosing the attack, “We are witnessing an attack by a nation with top-tier offensive capabilities.”

Furthermore, around 18,000 customer systems are estimated to have installed the malware from a ‘trusted upgrade’ of the company’s Orion product.

Forensic timelines even spotted the SolarWinds attackers running tests with planted code in the compromised supply chain in October 2019. It shows how much dwell time happened before official detection in December 2020.

How is it done?

By emulating Tactics, Techniques and Procedures (TTPs), leveraging currently known adversarial capabilities through capability abstraction techniques, and adapting to your organization’s environment network, a red team can effectively apply the same offensive methods real threats wield.

They can also leverage the same exploitable trust relationships in degrees comparable to that of a real adversary’s performance in a target organization’s network.

Point-in-time penetration testing is not enough to uncover complex attack chain breaches. Usually, in a vulgaris pentest, the only defined goal is to discover and patch exploitable technical vulnerabilities against a limited and often security-biased scope.

This then leaves flaws in organization processes, logical access control flows, and staff awareness checks widely unverified. With no strategic end goal in the scope, attackers know that a typical penetration test is limited when challenging an organization’s security biases.

This is precisely the fertile gray area where these highly targeted attacks seem to succeed and cause great havoc. From an adversarial perspective, it is about identifying and exploiting a target organization’s security biases. 

We have seen in breach case studies that adversaries ultimately have ambitious goals, and their impact, once objectives are achieved, is critical.

Your organization’s red team engagement efforts should also benefit from data-led goals and objectives aligned with an actual attacker’s real strategic intentions. Creating and developing favorable conditions to enact such engagement scenarios are also crucial for an organization’s software supply chain cyber defense strategy, especially if there are unique or legal requirements gaps regarding scope.

A simple example of such an engagement scenario that could save an organization millions in damages could be to assume that staff access is compromised and have red-teamers target and abstract adversarial capabilities against supply-chain related high-value assets, which typically for software would be:

• Source code repositories
• CI/CD pipelines
• Project documentations
• Credential access
• Software engineering staff awareness

Closing thoughts

Having implicit trust in software today is wrong and often occurs. There should always be a zero-trust concept.

Implicit trust in such areas is the principal cause of supply chain attacks. As a basis of security, there is no one-fits-all global solution for the vast array of available case studies across industries and organizations. 

Hence, a trial-by-fire red team challenge approach is necessary on top of all the existing defensive solutions and best practices deployed today to thwart such attacks.

In this article, we propose incorporating a continuous offensive security capability layer to your current software supply chain defense strategy. The described offensive security layer is led by the principal objective of achieving progressive compromise of your organization’s CI/CD pipelines before real attackers do so by leveraging the same techniques and capabilities that adversaries bring to the table. 

Capability abstraction techniques allow red teamers to borrow from the book of dozens of documented APTs and methods, which are collaboratively integrated and used against your organization’s supply chain to find breach opportunities during red team engagements.

Every time your red team capability program successfully identifies a breach opportunity, you can rest assured that they will inform you to apply necessary fixes and adjustments to your existing software supply chain defense strategy.

Nation-state-sponsored adversaries will not. It is our view that emphasizing the general adoption of objective based red team engagements against the wider industry’s software supply chains will significantly reduce the number of severe cyber breach incidents and damage costs projected by 2026. 

At CovertSwarm, we can help you secure your software supply-chain CI/CD pipelines by providing top-tier offensive security capabilities. Through our red team services, our Swarm of ethical hackers will relentlessly launch realistic cyber attacks across the full scope of your brand, using digital, physical and social engineering methods via a monthly subscription.