Home Blog Page 522

Let’s Encrypt ACME Certificate Protocol Set for Standardization

The open-source Let’s Encrypt project has been an innovating force on the security landscape over the last several years, providing millions of free SSL/TLS certificates to help secure web traffic. Aside from the disruptive model of providing certificates for free, Let’s Encrypt has also helped to pioneer new technology to help manage and deliver certificates as well, including the Automated Certificate Management Environment (ACME).

ACME is no longer just a Let’s Encrypt effort at this point in 2017 and is now being standardized by the Internet Engineering Task Force (IETF). The ACME protocol can be used by a Certificate Authority (CA) to automate the process of verification and certificate issuance.  

Read more at eWeek

Practical Networking for Linux Admins: Real IPv6

When last we met, we reviewed essential TCP/IP basics for Linux admins in Practical Networking for Linux Admins: TCP/IP. Here, we will review network and host addressing and find out whatever happened to IPv6?

IPv4 Ran Out Already

Once upon a time, alarms were sounding everywhere: We are running out of IPv4 addresses! Run in circles, scream and shout! So, what happened? We ran out. IPv4 Address Status at ARIN says “ARIN’s free pool of IPv4 address space was depleted on 24 September 2015. As a result, we no longer can fulfill requests for IPv4 addresses unless you meet certain policy requirements…” Most of us get our IPv4 addresses from our Internet service providers (ISPs), so our ISPs are duking it out for new address blocks.

What do we do about it? Start with bitter laughter, because service providers and device manufacturers are still not well-prepared, and IPv6 support is incomplete despite having more than a decade to implement it. This is not surprising, given how many businesses think computing is like office furniture: buy it once and use it forever (except, of course, for the executive team, who get all the shiny new doodads while us worker bees get stuck with leftovers). Google, who sees all and mines all, has some interesting graphs on IPv6 adoption. Overall adoption is about 18 percent, with the United States at 34 percent and Belgium leading at 48 percent.

What can we Linux nerds do about this? Linux, of course, has had IPv6 support for ages. The first stop is your ISP; visit Test IPv6 to learn their level of IPv6 support. If they are IPv6-ready, they will assign you a block of addresses, and then you can spend many fun hours roaming the Internet in search of sites that can be reached over IPv6.

IPv6 Addressing

IPv6 addresses are 128-bit, which means we have a pool of 2^128 addresses to use. That is 340,282,366,920,938,463,463,374,607,431,768,211,456, or 340 undecillion, 282 decillion, 366 nonillion, 920 octillion, 938 septillion, 463 sextillion, 463 quintillion, 374 quadrillion, 607 trillion, 431 billion, 768 million, 211 thousand and 456 addresses. Which should be just about enough for the Internet of Insecure Intrusive Gratuitously Connected Things.

In contrast, 32-bit IPv4 supplies 2^32 addresses, or just under 4.3 billion. Network address translation (NAT) is the only thing that has kept IPv4 alive this long. NAT is why most home and small businesses get by with one public IPv4 address serving large private LANs. NAT forwards and rewrites your LAN addresses so that lonely public address can serve multitudes of hosts in private address spaces. It’s a clever hack, but it adds complexity to firewall rules and services, and in my not-quite-humble opinion that ingenuity would have been better invested in moving forward instead of clinging to inadequate legacies. Of course, that’s a social problem rather than a technical problem, and social problems are the most challenging.

IPv6 addresses are long at 8 hexadecimal octets. This is the loopback address, 127.0.0.1, in IPv6:

0000:0000:0000:0000:0000:0000:0000:0001

Fortunately, there are shortcuts. Any quad of zeroes can be condensed into a single zero, like this:

0:0:0:0:0:0:0:1

You can shorten this even further, as any unbroken sequence of consecutive zeros can be replaced with a pair of colons, so the loopback address becomes:

::1

Which you can see on your faithful Linux system with ifconfig:

$ ifconfig lo
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host

I know, we’re supposed to use the ip command because ifconfig is deprecated. When ip formats its output as readably as ifconfig then I will consider it.

Be Quiet and Drink Your CIDR

Classless Inter-Domain Routing (CIDR) defines how many addresses are in a network block. For the loopback address, ::1/128, that is a single address because it uses all 128 bits. CIDR notation is described as a prefix, which is confusing because it looks like a suffix. But it really is a prefix, because it tells you the bit length of a common prefix of bits, which defines a single block of addresses. Then you have a subnet, and finally the host portion of the address. 2001:0db8::/64 expands to this:

2001:db8:0000:0000:0000:0000:0000:0000
_____________|____|___________________
network ID   subnet  interface address

When your ISP gives you a block of addresses, they control the network ID and you control the rest. This example gives you 18,446,744,073,709,551,616 individual addresses and 65,536 subnets. Mediawiki has a great page with charts that explains all of this, and how allocations are managed, at Range blocks/IPv6

2000::/3 is the global unicast range, or public routable addresses. Do not use these for experimentation without blocking them from leaving your LAN. Better yet, don’t use them and move on to the next paragraph.

The 2001:0DB8::/32 block is reserved for documentation and examples, so use these for testing. This example assigns the first available address to interface enp0s25, which is what is what Ubuntu calls my eth0 interface:

# ip -6 addr add 2001:0db8::1/64 dev enp0s25
$ ifconfig enp0s25
enp0s25   Link encap:Ethernet  HWaddr d0:50:99:82:e7:2b  
          inet6 addr: 2001:db8::1/64 Scope:Global

Increment up from :1 in hexadecimal: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, a, b, c, d, e, f, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 1a, 1b, and so on.

You can add as many addresses as you like to a single interface. You can ping them from the host they’re on, but not from other hosts on your LAN because you need a router. Next week, we’ll set up routing.

IPcalc

All of these fine hexadecimal addresses are converted from binary. Where does the binary come from? The breath of angels. Or maybe the tears of unicorns, I forget. At any rate, you’re welcome to work these out the hard way, or install ipcalc on your Linux machine, or use any of the nice web-based IP calculators. Don’t be too proud to use these because they’re lifesavers, especially for routing, as we’ll see next week.

Learn more about Linux through the free “Introduction to Linux” course from The Linux Foundation and edX.

How to Install Linux on a Chromebook (And Why You Should)

Chromebooks are one of the most secure devices you can give a non-technical end user, and at a price point few can argue with, but that security comes with a privacy trade off: you have to trust Google, which is part of the NSA’s Prism programme, with your data in the cloud.

Even those who put their faith in the company’s rusty “don’t be evil” mantra may find Chromebook functionality limiting—if you want more than Google services, Netflix, some other Web apps, and maybe the Android app store, then you’re out of luck.

Geeky users willing to engage in some entry-level hackery, however, can install Linux on their Chromebook and unleash the Power of Torvalds™.

Read more at Ars Technica

DNS Infrastructure at GitHub

At GitHub we recently revamped how we do DNS from the ground up. This included both how we interact with external DNS providers and how we serve records internally to our hosts. To do this, we had to design and build a new DNS infrastructure that could scale with GitHub’s growth and across many data centers.

Previously GitHub’s DNS infrastructure was fairly simple and straightforward. It included a local, forwarding only DNS cache on every server and a pair of hosts that acted as both caches and authorities used by all these hosts. These hosts were available both on the internal network as well as public internet. We configured zone stubs in the caching daemon to direct queries locally rather than recurse on the internet. We also had NS records set up at our DNS providers that pointed specific internal zones to the public IPs of this pair of hosts for queries external to our network.

Read more at GitHub

Encryption Technology in Your Code Impacts Export Requirements

US export laws require companies to declare what encryption technology is used in any software to be exported. The use of open source makes complying with these regulations a tricky process.

US Export Requirements

The regulations on US software exports come from the US Commerce Department’s Bureau of Industry and Security (BIS). The specific regulations are called Export Administration Regulations (EARs). The restriction of encryption is based in national defense concerns: we don’t want bad guys to be able to hack into our secret communications, nor prevent us from cracking into theirs. 

The specifics of these regulations are complex and belong in the realm of experts. The basics are that you need to tell the BIS what encryption is in any software you export, though it restricts only strong cryptography, with particular sensitivity to a small number of bad actor nation states. The agency is serious about the requirements and has been known to enforce them, notably fining Wind River $750,000 in 2014 (despite Wind River’s voluntarily disclosing the issue they had discovered themselves).  

Read more at Black Duck

Why Infrakit & LinuxKit Are Better Together for Building Immutable Infrastructure?

Let us accept the fact – “Managing Docker on different Infrastructure is still difficult and not portable”. While working on Docker for Mac, AWS, GCP & Azure, Docker Team realized the need for a standard way to create and manage infrastructure state that was portable across any type of infrastructure, from different cloud providers to on-prem. One serious challenge is that each vendor has differentiated IP invested in how they handle certain aspects of their cloud infrastructure. It is not enough to just provision n-number of servers;what IT ops teams need is a simple and consistent way to declare the number of servers, what size they should be, and what sort of base software configuration is required.

Also, in the case of server failures (especially unplanned), that sudden change needs to be reconciled against the desired state to ensure that any required servers are re-provisioned with the necessary configuration. Docker Team introduced and open sourced “InfraKit” last year to solve these problems and to provide the ability to create a self healing infrastructure for distributed systems.

Read more at Collabnix

Viewing Linux Output in Columns

The Linux column command makes it easy to display data in a columnar format — often making it easier to view, digest, or incorporate into a report. While column is a command that’s simple to use, it has some very useful options that are worth considering. In the examples in this post, you will get a feel for how the command works and how you can get it to format data in the most useful ways.

By default, the column command will ignore blanks lines in the input data. When displaying data in multiple columns, it will organize the content by filling the left column first and then moving to the right. For example, a file containing numbers 1 to 12 might be displayed in this order:

Read more at Network World

Cloud Native Apps and Security: The Case for CoreOS Rkt and Xen

CoreOS’s rkt started at the beginning of 2014 as a security-focused alternative to Docker. The project aimed to create a signature verification of cloud-native apps by default; the intention was to guarantee the integrity of the apps. It also stepped away from the central-daemon design of Docker, which requires root privileges for all operations. By contrast, the rkt process is short-lived, limiting the chances of being exploited, and some of rkt commands can be executed as unprivileged user.

The project has come a long way since it was conceived. It is stable, fully featured, and it supports a variety of ways to fetch and start cloud-native apps with security being top of mind. For example, it can download apps from the Docker registry and use virtualization to run them.

Rkt champions open standards and supports the Open Container Initiative image and runtime specifications. A couple of months ago, it was accepted into the Cloud Native Computing Foundation, becoming a member of the same family as Kubernetes, the popular container orchestration service. Rkt already has excellent support for Kubernetes, and it will strengthen further now that they live under the same roof.

Why Rkt Works So Well: It’s The Architecture

The primary benefit of rkt is that it is versatile. Its versatility stems from its architecture, which is based on multiple stages of execution: stage0, stage1, and stage2:

  • Stage0 is responsible for readying the images and is implemented by the rkt executable.

  • Stage1 is in charge of creating isolated environments to run cloud-native apps. Stage1s are distributed in Application Container Image format (also known as ACI), which is a tarball containing a rootfs and a JSON manifest. It is the same format used for cloud-native apps.

  • Stage2 is the environment in which the applications actually run. 

Taking a lesson from Kubernetes, the basic execution unit of rkt is a pod: a small set of cloud-native apps to be run in a shared context. Typically, a pod is just a couple of apps, for example, a server app and a log parsing app. The log parsing application needs access to the logs of the other app; hence, the two apps share filesystem access.

When the user executes rkt run to start a pod, rkt unpacks the stage1 tarball, sets up the stage2s’ rootfs at a known location under the stage1 filesystem hierarchy, and runs a stage1 application with the right arguments. The stage1 application to run is specified in the stage1 manifest. The stage1 binary takes charge of setting up a fresh new environment, then runs the stage2 applications.

The beauty of this architecture is that stage1s are entirely independent and self-contained. Developers can implement new stage1s easily. They can be maintained, built, and shipped separately from rkt. Today, rkt supports five in-tree stage1s, plus two out-of-tree, including a stage1 based on Linux namespaces named coreos, and a stage1 based on KVM. End-users are given the choice of multiple stage1s with different trade-offs; they can pick the best for their use-cases at runtime, with a simple command line option.

The industry has come a long way since the early days of Docker, when many people confused cloud-native apps with Linux namespaces, because they were both called containers. Linux namespaces are only one of the many technologies to run applications. Similarly, cloud-native apps are packaged according to the ACI format, which is only one of many ways to package applications binaries. The two technologies are orthogonal. The distinction between them is extremely stark in rkt. 

Xen Joins the Party

A couple of weeks ago, the rkt community gained stage1-xen, a new stage1 based on the Xen Project hypervisor. It is still in its very early days, but it is a good proof of concept. Xen Project offers a few unique properties, not just in terms of technology, but also in terms of community and processes.

Xen Project is known as the enabler of many strong isolation and privilege separation architectures. Projects like Qubes OS and OpenXT, aimed at highly secure environments, take the security by compartmentalization approach, using the Xen Project hypervisor to create multiple isolated compartments. Each workload runs on a separate virtual machine. Infrastructure components, such as the network stack and the network drivers, can also be moved into their own separate VMs, named driver domains. Even if an attacker manages to penetrate and assume control of a driver domain, the intruder still does not gain full system access.

The Xen stage1 enables users to take advantage of rkt’s easy to use and powerful app management features, together with the Xen Project’s security and isolation properties. It creates a separate, secure by default, Xen virtual machine for each pod.

Configuring Linux namespaces for isolation is hard; it is a daunting task at any scale. SELinux is the top technology to do it, but it has a steep learning curve, and often end-users disable it. It is hard to believe that the first completion suggestion for “how to disable” on Google Search is actually “selinux.” As companies are redesigning their software stacks around microservices, they’ll benefit from a Xen Project solution which is secure and doesn’t need additional settings to increase isolation.

Xen is most often associated with the largest public clouds in production, but the target of this project is not limited to servers. In fact, cloud-native apps are becoming the new way of packaging and distributing applications across all market segments. Stage1-xen will be of great help to developers in embedded environments, such as the automotive industry, where higher security standards are to be upheld. It will allow them to download and deploy new apps to vehicles, keeping them strongly isolated from the critical functions of the car.

Xen and Its Proclivity for Cloud Computing

There are many reasons why Xen is a great hypervisor for cloud-native applications; one of them is that Xen can run anywhere, from the latest and greatest physical servers to the smallest Amazon AWS instances. Let’s start by looking at virtualization technologies to understand how this is possible.

Xen offers two virtual machine types on Intel and AMD processors: PV and HVM guests. The Xen stage1 uses PV guests because they are lightweight and they don’t require any hardware emulation or additional processes on the host. Also, they have short boot times as they don’t run any guest firmware (i.e., there is no UEFI or Seabios to be run inside the virtual machine). They are a good match for cloud-native apps.

A fundamental characteristic of PV guests is that they don’t require hardware virtualization extensions. Intel calls them VT-x, while AMD named them AMD-V. They were introduced around 2006. All modern x86 machines support them, but cloud instances do not.

Although both Xen and KVM can create virtual machines with a virtual version of VT-x and AMD-V, cloud providers do not enable this feature. As a consequence, Amazon and Google Cloud instances look like pre-2006 hardware: they have neither VT-x nor AMD-V. Thus, it is not possible to create a nested KVM virtual machine on top of an Amazon AWS instance, but it is possible to start a nested Xen PV guest in the same environment because it doesn’t require virtualization extensions. With stage1-xen, Rkt users gain the ability to execute cloud-native apps as virtual machines on top of AWS and Google Cloud, the same way they do today with the default coreos stage1.

Beyond the Technicalities: The Security Process

Besides the technical features, Xen Project has a strong security track record and a fully transparent security policy that supports responsible disclosure.

Security fixes are easy to track, apply and deploy. Stable trees are maintained for two years. It is possible to patch productions systems before the public disclosure date when a fix doesn’t expose technicalities that could introduce the risk of re-discovery of the vulnerability. Security management is one of the top reasons for choosing the Xen Project hypervisor, which makes it a great fit for a security-focused project like rkt.

Stage1-xen is still in its infancy, and we need your help in making it fully supported and ready for primetime.

If you are interested in cloud-native apps and security, join the community and take the opportunity to shape its future. If you are located in or near Budapest, Hungary, I’ll be talking more about this topic during the Xen Project Developer and Design Summit happening July 11 – 13, 2017.

Hardware Is the New Software

Hardware is the new software Baumann, HotOS’17

This is a very readable short paper that sheds an interesting light on what’s been happening with the Intel x86 instruction set architecture (ISA) of late. We’re seeing a sharp rise in the number and complexity of extensions, with some interesting implications for systems researchers (and for Intel!). We’re also seeing an increasing use of microcode blurring the role of the ISA as the boundary between hardware and software.

We argue that these extensions are now approaching software-like levels of complexity, yet carry all the attendant drawbacks of a hardware implementation and the slow deployment cycle that implies. We suspect that the current path may be unsustainable, and posit an alternative future with the ultimate goal of decoupling new ISA features from the underlying hardware.

Read more at The Morning Paper

Hallmarks of a Good Technical Leader

I recently sat down with Camille Fournier, the head of Platform Engineering at Two Sigma, to talk about what constitutes great technical leadership and how organizations can foster it. Here are some highlights from our chat.

How do you define technical leadership (as opposed to leadership in general)?

Technical leaders don’t just generically inspire people to do things, but are capable of communicating with technical stakeholders and engineers in language that they understand. Technical leadership is about understanding the technical context under which decisions are being made, and asking questions to help make sure the right decisions are being made given the technical concerns.

Read more at O’Reilly