A software environment’s attack surface is defined as the sum of points in which an unauthorized user or malicious adversary can enter or extract data. The smaller the attack surface, the better. We recently sat down with Doug Goldstein (https://github.com/cardoe or @doug_goldstein) to discuss how companies can use hypervisors to reduce attack surfaces and why the Xen Project hypervisor is a perfect choice for security-first environments. Doug is a principal software engineer at Star Lab, a company focused on providing software protection and integrity solutions for embedded systems.

Linux.com: Tell us a little bit about what your company does and your area of expertise?

Doug Goldstein: Star Lab is a software security provider dedicated to researching, developing, testing, and delivering embedded security solutions for both commercial and government customers. We address at-rest, boot, and runtime system protections even in the face of successful privilege escalation attacks. Star Lab maintains a Xen-based and security-focused virtualization product called Crucible that targets embedded markets.

As for me, I’ve been a Gentoo Linux developer for about 15 years, and have been interested in virtualization for some time. I started my foray into virtualization with KVM for test environments, and have contributed back to that community. I’m a contributor for the Xen Project and have worked to make a hypervisor modular at compile time through a project called Kconfig, which was borrowed from the Linux kernel. Kconfig allows developers to create a more lightweight hypervisor, which reduces attack surface and is beneficial in security-first environments, microservice architectures, IoT, and industries with heavy compliance and certification requirements (such as the automotive sector). Kconfig was released as part of Xen Project Hypervisor 4.7.

Recently, my coworkers and I have been looking more closely into IoT security. With an ever-increasing number of connected devices being brought to market, there are more vectors than ever for attackers to get into remote systems and access sensitive data—even with protective measures such as firewalls in place. For example, someone could attack your laptop through your smart TV. Given the current security landscape, the same types of software protections and system integrity solutions we’ve been creating for the government are now recognized as necessary for consumers and B2B.

Here are a few other technologies that started out in the military, but trickled to the consumer realm, if you like this type of stuff.

Linux.com: When it comes to securing devices, what is the approach that you use?

Goldstein: Using a multi-layered protection and detection approach is the only way to ensure systems stay secure. Many organizations approach security with a network-based intrusion detection system or a firewall and believe that is enough. We believe that security must take a more holistic, proactive approach. If you only have an intrusion detection system, you can only see attacks at a very high level. For example, you might be able see someone attempting to exploit a service, but what if they somehow got valid login credentials? To begin implementing truly secure architectures, you first need to lockdown and control capabilities in the edge systems. This is where the hypervisor comes into play.

A hypervisor is an important  piece in the multi-layered security approach. By separating system components into different VMs, an attacker would have to compromise each VM to modify or access sensitive data, versus compromising one service and having access to all the data. It is also possible to use redundant VMs using technologies like COLO (Course Grain Lock Stepping). This allows two VMs to process the same input and ensure they agree on a valid response, thereby preventing an attacker from permanently modifying the data.

Also, hypervisors can provide multiple levels of privilege so that a service with sensitive data could run as a VM alongside a service with less sensitive information, and they would never be able to see or modify each other’s data. This is one way we are able to reduce costs in government projects that have data at different classification levels. We allow them to use a single system with one or more VMs, instead of two physical systems.

Finally, by using a hypervisor, you can prevent certain classes of persistent attacks because some of these attacks rely on direct access to hardware, flash storage, or BIOS memory. By running services inside constrained VMs, an attacker has no direct access to the system hardware without first defeating the hypervisor.

Linux.com: Why did you choose the Xen Project hypervisor, and how does it differ in terms of security compared to something like KVM?

Goldstein: One of the main reasons why we chose to focus on the Xen Project (and likely why many other security-first projects choose the Xen Project over KVM) is that its architecture allows for strong isolation and privilege separation. This means that the Xen Project hypervisor is able to live separately from the Linux Kernel and the Linux Kernel is able to be separated into less privileged pieces. For example, if there is an attack on the Linux Kernel, it is not going to impact Xen like it impacts KVM.

Another example of this is network card drivers that can be separated into their own driver domains, such as what OpenXT and Qubes do with Xen. The whole idea is, even if an attacker can exploit the driver, the attacker does not gain full access to the hypervisor. This isn’t the case with KVM, where a single kernel or driver compromise can undermine the entire system security posture.  

Linux.com: The Xen Project serves a lot of different use cases. What can you do to make it better serve a security use case?

Goldstein: As mentioned above, Kconfig gives you the ability to make the Xen Project hypervisor more lightweight, thereby eliminating attack surfaces. For example, with Kconfig you can take out some of the migration features that might be essential in cloud environments, but don’t make sense for security-first environments.

Other examples include VM introspection and fine-grained access controls. VM introspection allows you to check the state of your VM in real time without having to run any code inside the VM. This allows you to ensure that no one has modified a critical piece of software running inside the VM. Fine-grained access controls that XSM FLASK provides allow or restrict communications between VMs or between the VM and the hypervisor. This provides a way to reduce the attack surface of the VMs and the hypervisor.

Linux.com: How about containers? How would containers fit into this?

Goldstein: Some of the people that we work with wanted to put containers into their infrastructure because they believed it was more secure. On one hand, it is a little better because the software is carved up into separate containers, making it easier to ship updates without affecting the other software running on the system. In this sense, containers can be good from a security standpoint.

In many cases, however, people don’t actually deliver these updates and fixes. People build up containers, but don’t update the software within the containers. For example, there could be multiple copies of OpenSSL in different containers that are all out of date. In this scenario, even though a system administrator updated packages in a server’s OS, the machine would still have vulnerabilities. Containers without good software management practices can actually reduce the system’s overall security posture.

It might be interesting to note at this point that containers do not address kernel-level attacks. Containers limit the attack surface of an individual service, preventing one service from affecting other services on the machine. However, containers are less secure than virtual machines since the kernel has a much larger attack surface than a hypervisor and is more vulnerable to privilege escalation attacks—possibly leading to an escape from the container. The smaller attack surface of the hypervisor decreases the probability that a privilege escalation attack will allow an attacker to break out of a virtual machine and affect other VMs on the system.

Linux.com: Which areas do you think could be improved from a security standpoint currently?

Goldstein: One area I would love to see more interest in, from a Xen Project perspective, is XSM FLASK. It is not the default access control mechanism in the Xen Project hypervisor at present.  Many people within the community recognize the benefits of switching to XSM FLASK, but there is a lot of inertia around the existing model. There are issues in the current model that can be elegantly solved by creating the right policies using XSM FLASK. Focusing on using XSM FLASK and enhancing Xen’s default policy, rather than adding more knobs to the existing access control model, could potentially benefit the entire Xen Project community.

Additionally, a lot of maintainers in the security group have overlapping use cases that specifically focus on cloud hosting environments. It would be great to get more members from the security community to engage the Xen Project and even potentially become maintainers focused on use cases other than cloud-hosted environments. For example, virtualization is a great solution for addressing functional safety, multi-level access, system integrity, and cybersecurity problems in the embedded systems space. This broadening of the community would allow a larger set of use cases to be covered by the Xen Project, in a single open source project, rather than siloed within several separate projects.

If you want to learn more about this topic, check out Doug’s presentation at the Xen Project Developer Summit below: