Click up chevron icon

The evolution of endpoint security: Lessons from the past and why it must evolve again

Learn how endpoint security has evolved alongside changing threats and IT landscapes, why modern environments demand a new approach, and what comes next.

TL;DR

  • From prevention to detection: Endpoint security has evolved from traditional antivirus, next-generation antivirus, and Endpoint Protection Platforms (focused on prevention) to include detection solutions like Endpoint Detection and Response (EDR), allowing organisations to monitor and respond to active threats.
  • The turning point: The widespread adoption of virtualisation and cloud computing expanded the definition of an "endpoint" and agentless solutions emerged in response to the limitations of agent-based ones. This sparked a debate on the merits of agent-based vs. agentless models.
  • Evolving threats: Cyber threats have grown more sophisticated, with techniques like kernel-level exploits and rootkits becoming more common, and a growing trend of EDR bypassing and tampering. This is pushing existing endpoint security solutions to their limits.
  • The case for hypervisor-based detection: With the increasing prevalence of virtualised environments, hypervisor-based threat detection offers a new, effective approach that provides better isolation, real-time insights, and stronger protection against modern threats.

Introduction

Endpoints have always been prime targets, and often the starting point, for cyber attacks. If an attacker can compromise an endpoint, they gain a foothold to move deeper into an organisation’s network and gain access to other endpoints where critical data is stored, processed, and accessed. That’s why endpoint security is a critical component of cybersecurity strategies.

Over the years, endpoint security has evolved in response to the threat landscape and IT infrastructure changes – and it must evolve, again. Attackers have become more sophisticated and cyber threats aren’t what they used to be. At the same time, modern IT environments present new challenges that previous security models were never built to handle. The way we define an “endpoint” today has expanded far beyond physical devices and is now stretching to cover virtual machines, cloud workloads, and containers, which are part of dynamic, fluid environments that barely resemble the static infrastructure that initial endpoint security solutions were built to protect.

To understand where we need to go, we first need to understand how we got here. In the following sections, we'll retrace the evolution of endpoint security from the early days of simple antivirus through to today's solutions. We'll examine why each new generation of tools emerged, what problems they solved, what challenges they faced, and what comes next.

Antivirus as the first wave of endpoint security tools

With the increasing popularity of personal computers in the 1980s, and the growth of computer networks and the internet in the late 1980s and early 1990s, there was a growing need to detect and remove viruses from computers to protect both personal users and organisations.

To address this, antivirus (AV) emerged as the first line of defence and became a standard security measure for decades. Traditional AV solutions rely on signature-based detection – identifying known malware by matching it against a database of threat signatures. While effective at blocking known threats, this approach soon faced a significant limitation: it couldn’t stop new or modified malware that didn’t yet have a signature – and cybercriminals were quick to take advantage of that.

The evolution to Endpoint Protection Platforms and Next-Generation Antivirus

In response to the growing doubts about the effectiveness of traditional antivirus and the growing complexity of enterprises IT infrastructures, Endpoint Protection Platforms (EPP) emerged in the late 2000s. EPPs combine multiple security measures, including antivirus, firewall, and other security features into a single, integrated solution, aiming to provide a more holistic approach to endpoint security. However, early EPPs still relied heavily on signature-based detection, as a continuation of the antivirus technology that preceded them.

This created security gaps. Attackers developed techniques to evade detection, such as polymorphic malware – a type of malicious software that can constantly change its code or appearance so that it can’t be recognised through signature-based detection. It became clear that organisations needed a new approach that didn’t rely solely on known signatures.

Came in Next-Generation Antivirus (NGAV), which emerged as a more advanced form of AV technology and became prominent in the 2010s. Instead of relying only on signature matching, NGAVs use machine learning, behavioural analysis, and heuristic techniques to detect suspicious activity, allowing them to identify and block both known and unknown threats.

When introduced, EPPs and NGAVs marked a significant improvement over traditional AVs. Primarily focused on prevention, they are designed to detect and block threats before they can execute and cause damage. It is their main strength, but also their main limitation: they don’t provide any form of visibility into what’s happening when an attack manages to break through.

From prevention to detection with Endpoint Detection and Response

IT environments grew more complex and more connected, increasing the attack surface. At the same time, malware continued to become more advanced. AVs, NGAVs, and EPPs were unable to prevent all attacks. Organisations needed more. They needed visibility into what was happening on their endpoints, and a way to detect, investigate, and respond to threats that had already infiltrated their systems.

This led to the creation of Endpoint Detection and Response (EDR) solutions. Unlike previous endpoint security solutions, EDRs focus on detection, continuously monitoring endpoint activity to detect suspicious behaviours in real time. By installing software agents directly on endpoints, which allow them to collect and analyse endpoint data, EDRs enable security teams to identify threats that have successfully infiltrated an organisation, investigate incidents, and respond swiftly to reduce the potential for damage and data loss. Working alongside prevention solutions as part of a layered security strategy, EDRs were popularised in the early 2010s and became a cybersecurity staple.

But, just when you might think organisations finally had their endpoint security arsenal sorted out, another foundational change happened: cloud computing and virtualisation went "from niche to everywhere”.

The turning point: Virtualisation, cloud computing, and the impact on endpoint security

The early 2010s marked a significant transformation in how businesses and individuals accessed and utilised IT resources, characterised by the mainstream adoption of cloud computing and virtualisation technologies, which became foundational to today’s modern IT infrastructure.

This ‘boom’ significantly impacted endpoint security in several ways:

  • The definition of endpoints expanded beyond physical devices to include virtual machines (VMs), cloud workloads, and containers.
  • Organisations saw a significant increase in the number of endpoints, creating a larger attack surface for threat actors to exploit.
  • IT infrastructures became more dynamic and the way endpoints are deployed and managed changed, introducing new difficulties for defenders and requiring endpoint security to adapt to rapidly changing environments.

Driven by the need to secure these increasingly complex and distributed IT infrastructures, EDRs solutions evolved to support deployments across a wider range of endpoints, whether physical or virtual. At the same time, new security solutions built purposefully for the cloud and virtualised environments started to emerge, such as Cloud Workload Protection Platforms (CWPPs), which provide runtime visibility and protection for a range of workloads.

However, in line with traditional endpoint security methods, these solutions were primarily agent-based, and the limitations of the agent-based model quickly became apparent in these new environments.

The challenges for agent-based detection and response

Agent-based security tools face several limitations in cloud and virtualised environments, since they rely on deploying agents on each individual machines. First of all and most importantly, a critical security limitation is that the presence of agents themselves can potentially be exploited by attackers and provide a false sense of security. Agents are searched for and can be bypassed, or even completely neutralised, and the data can be tampered with. Moreover, agent-based security tools pose several operational challenges. Installing, configuring, and monitoring agents is often impractical due performance constraints, scalability issues, and operational overhead.

To address some of these challenges, EDRs evolved to work more effectively in virtualised environments, incorporating advancements such as deploying lightweight agents that can operate efficiently within the constrained resources of a VM, or leveraging cloud computing for more intensive analytical tasks while maintaining on-device capabilities for rapid threat detection on the endpoint itself.

While these developments have improved agent-based solutions, they have not fully addressed all the inherent challenges they face.

The emergence of the agentless detection and response model

To close the gaps left by agent-based tools, a new approach emerged: agentless detection and response. Rather than relying on software agents installed on each endpoints, these solutions rely on other data collection methods, such as network traffic analysis, system logs, image scanning, configuration assessment, centralised scanning, and API integrations, to detect and respond to threats. By removing the need for agents, agentless solutions provide visibility into short-lived workloads that agent-based tools struggled to cover, reduce performance overhead, and eliminate the operational challenges of managing security agents at scale. With such advantages, the agentless model became popular in the late 2010s, especially amongst organisations with large, complex, and dynamic infrastructures.

However, while agentless detection improves security coverage and reduces operational overhead, it lacks the deep, endpoint-level visibility and real-time capabilities that agent-based solutions can provide.

The discussion centred around the trade-offs between the detailed, real-time insights offered by agent-based solutions versus the simplicity and scalability of agentless approaches, ultimately developed into a hot debate around the relative merits of each approach for security monitoring.

Agent-based or agentless? From competing to combined approaches

As organisations grappled with the challenges of securing increasingly complex and dynamic environments, the ‘agent-based vs. agentless security debate’ gained significant traction in the early 2020s and reached its peak around 2023-2024.

While the debate is still ongoing, many have started to realise that both approaches have their own strengths and weaknesses, and that a combination of the two could balance comprehensive security with operational efficiency. Security vendors that were initially providing fully-agentless solutions started to develop “runtime sensors” (in other terms, agents) and, in practice, many organisations are now opting for a hybrid approach, leveraging both agent-based and agentless solutions.

While each threat detection model on its own and this hybrid approach have their benefits, both agentless and agent-based security models (as they are today) face a major security limitation: the issue of isolation and comprehensive protection against compromised operating systems (OS). This is crucial because:

  • Agentless systems often lack granular visibility into individual VMs and have limited real-time visibility, leaving gaps in runtime protection;
  • But with agent-based systems, if an attacker gains control of the OS, they can potentially disable, bypass, or even compromise, the security agent.

This has led some in the industry – from emerging startups to leading security players – to explore a different approach: hypervisor-based threat detection.

The emerging paradigm: Hypervisor-based threat detection

The idea behind hypervisor-based threat detection is simple but powerful: instead of relying on security solutions embedded within the endpoint, this approach shifts monitoring to the virtualisation layer – the hypervisor itself. By operating from outside the workload, hypervisor-based security establishes an isolated vantage point, one that attackers cannot easily manipulate.

At the same time, hypervisor-based security is inherently agentless, requiring no software installation. This removes the management complexity, and visibility gaps associated with agent-based security, while still providing granular, real-time insight into activities – something traditionally achievable only with agents.

While still an emerging approach, hypervisor-based threat detection holds clear potential. It offers a unique level of isolation, making it impracticable for attackers to disable or tamper with the security mechanisms. It is also capable of detecting sophisticated techniques that current solutions may struggle to address, ensuring both in-depth security monitoring and operational efficiency in virtualised environments. However, as with any new approach, hypervisor-based threat detection comes with its own trade-offs – something that deserves a deeper discussion at another time.

Another important point to note is that this approach will not be replacing all existing endpoint security methods. Physical devices and traditional endpoints will still require conventional security solutions. Rather, hypervisor-based threat detection emerges as a response to the reality that more and more endpoints and workloads today exist in virtualised environments, where security can and should be approached differently.

Why now? The urgency to rethink endpoint security strategies in virtualised environments

To detect an attack occurring at a specific level, security must be positioned at a different level. This principle has never been more relevant than it is today.

Cyber threats have evolved far beyond what was seen two decades ago. Low-level threats – i.e., sophisticated threats that operate at the lowest and most privileged level of the operating system, such as kernel-level exploits and rootkits – are becoming increasingly common, and existing detection models were not purposefully built to handle them. Adversaries are also refining their evasion techniques, with a growing trend of EDR bypassing and EDR tampering. As modern malware becomes more sophisticated, the likelihood of undetectable threats slipping through traditional defenses increases by the day.

What’s more, virtualisation is now the backbone of modern computing, with virtual machines, cloud workloads, and containers running on top of hypervisors. Yet many organisations continue to rely on retrofitted agent-based solutions and agentless solutions that come with well-documented limitations. But virtualisation itself offers an opportunity to rethink security. Instead of forcing outdated security models onto modern infrastructure, we can leverage the hypervisor not just as an infrastructure layer but as a security layer – one that provides better visibility, stronger isolation, and more effective detection against advanced and evasive threats.

By addressing the weaknesses of the current security approaches and aligning with the way modern infrastructures are actually built, hypervisor-based threat detection presents a highly relevant and fitting solution for today and tomorrow’s threat landscape.

Conclusion

History has shown: cybersecurity must evolve alongside the environments it protects. Endpoint security has already undergone multiple evolutions: from antivirus to NGAV, from EDR to more cloud-native models, from agent-based to agentless solutions, and now toward hybrid approaches. Each step has been driven by necessity, since adversaries refined their tactics and IT infrastructures evolved. Now, the next step is emerging: hypervisor-based threat detection.

This is not about blindly replacing existing security models, but about ensuring that detection and response strategies align with the environments they protect. Virtualised environments introduce a new security paradigm, where hypervisor-based threat detection offers a unique opportunity to establish a stronger foundation – one that works with modern IT infrastructure rather than against it. The question for organisations today is simple: Is your security strategy keeping up with the environments you are securing and the threats you need to defend against?

As Darwin’s theory suggests, adaptability is key. Organisations that fail to adapt their security strategies risk being outpaced by both technological advancements and increasingly sophisticated adversaries. Now is the time to rethink security approaches and explore solutions that align with today’s IT realities – before the next wave of threats forces the change upon us.