Switching from Solaris to Linux has become much easier in the last two years, with Linux developments in ZFS, Zones, and DTrace. I’ve been contributing (out of necessity), including porting my DTraceToolkit tools to Linux, which also work on BSD. What follows are topics that may be of interest to anyone looking to migrate their systems and skillset: scan these to find topics that interest you.
ZFS
ZFS is available for Linux via the zfsonlinux and OpenZFS projects, and more recently was included in Canonical’s Ubuntu Linux distribution: Ubuntu Xenial 16.04 LTS (April 2016). It uses a Solaris Porting Layer (SPL) to provide a Solaris-kernel interface on Linux, so that unmodified ZFS code can execute.
My company uses ZFS on Linux in production, and I’ve been the go-to person for deep ZFS problems. It feels largely the same, except kstats are in /proc/spl/kstat/zfs/arcstats, and I debug it with Linux tracing tools instead of DTrace (more on that next). There have been some issues on Linux, but overall it’s been ok, especially given how hard we push ZFS. We’ve used it for our container hosts (codename Titus) that do frequent snapshots, use send/recv, etc.
As an engineering leader, I value trust and believe that individual contributors should be involved in architectural and high-level technical decision making. I consider every line of code to be a decision made on behalf of someone else (including your future self), and having a fast-growing distributed team makes technical decision making particularly difficult to manage.
In the early days of building ride-sharing app Ride, we went from three to more than 25 members, across product, design, and engineering, in the first six months. We were tasked with the challenge of taking an early prototype for a carpooling platform and bringing it to life on the web, iOS, and Android. To make things more fun, we were also distributed across the United States, Mexico, Colombia, Brazil, Argentina, and Ireland.
In the process of building our apps, I received a private Slack message from “a not very happy front-end engineer,” who asked:
Why was the data dashboard built using React if our front-end stack is based on Ember?
This made me quickly realize a few important things I shouldn’t have missed:
This is an expanded version of my talk at NginxConf 2017 on September 6, 2017. As an SRE on the Dropbox Traffic Team, I’m responsible for our Edge network: its reliability, performance, and efficiency. The Dropbox edge network is an nginx-based proxy tier designed to handle both latency-sensitive metadata transactions and high-throughput data transfers. In a system that is handling tens of gigabits per second while simultaneously processing tens of thousands latency-sensitive transactions, there are efficiency/performance optimizations throughout the proxy stack, from drivers and interrupts, through TCP/IP and kernel, to library, and application level tunings.
Disclaimer
In this post we’ll be discussing lots of ways to tune web servers and proxies. Please do not cargo-cult them. For the sake of the scientific method, apply them one-by-one, measure their effect, and decide whether they are indeed useful in your environment.
All it takes is a fork from the main branch and a re-branding of the code, and the next thing you know, there’s a hidden threat in your software. Here’s how to protect against it.
“One of the aspects of open source is that it can be forked,” said Tim Mackey, the Technical Evangelist for BlackDuck Software. “If you look at GitHub today and look at the OpenSSL project, you’ll see that over 2500 or 2600 different OpenSSL forks have occurred,” If a vulnerability in the OpenSSL system occurs, as it did when the Heartbleed bug rose to fame, only the mainline, unforked version of the project will be tagged as being problematic. If the Docker container you downloaded is using a forked version of a piece of open source software, or your cloud computing stack uses a highly customized derivative, you may very well have a hidden threat buried within your system that you won’t be able to identify before hackers identify it for you.
What happens when you take Ubuntu 17.10, a new desktop interface (one that overlays on top of KDE), snap packages, and roll them all up into a pseudo rolling release? You get Nitrux. At first blush, this particular Linux distribution seems more of an experiment than anything else — to show how much the KDE desktop can be tweaked to resemble the likes of the Elementary OS or MacOS desktops. At its heart, however, it’s much more than that.
First and foremost, Nitrux makes use of snap packages; so installing software is handled a bit differently than the norm. Even though Nitrux is based on Ubuntu, apt install isn’t what you want to use (although it is available).
I’m getting ahead of myself. This distro focuses very much on the GUI — so the GUI should be the route you take. Good thing Nitrux includes a GUI software installer tool for that purpose. That Nitrux uses snaps is good and bad, and it’s the bad that will put users off faster than the good.
Again, I’m getting ahead of myself.
Let’s first talk about what Nitrux is. This particular take on the Linux desktop is focused on the portable, universal nature of snap packages and makes use of a unique desktop, called Nomad, which sits atop KDE Plasma 5. It’s minimum requirements are:
2.66 GHz quad-core CPU or better.
4 GB system memory.
256 MB video memory and OpenGL 2.0 support.
4.29 GB of free hard drive space.
On that 4.29GB of free hard drive space, I installed Nitrux as a VirtualBox VM with 10GB of space. Upon installation, I installed the LibreOffice snap package, only to find out I was then out of space. I don’t know about you, but no LibreOffice installation I’ve ever done takes 5GB of space. I say this, only so you’ll be aware, should you opt to text Nitrux via VirtualBox—give that virtual disk about 20GB of space.
The Nomad desktop (Figure 1), will feel instantly familiar.
Figure 1: The default Nitrux desktop.
The desktop includes a dock, a system/notification tray, a quick search tool (Plasma Search), and an app menu. Of all the elements on the desktop, it’s the Plasma Search tool that will appeal to anyone looking for an efficient means to interact with their desktops. With this tool, you can just start typing on a blank desktop to see a list of results. Say, for example, you want to open LibreOffice writer; on the blank desktop, just start typing “libre” and related entries will appear (Figure 2).
Figure 2: The menu search tool in action.
The search feature is also capable of enabling/disabling various plugins, to add extra functionality. For example, you can include Bookmarks, web shortcuts, terminal applications, power management, and more to the desktop search (this is done through System Settings > Search > Plasma Search).
Take a step back
Although the Nitrux desktop might well be something every new user (even to Linux) could get up to speed with quickly, getting to the point of usage can be a bit confusing. Why? First off, when booting up the live instance, you are asked for a password, but never told what it is. Said password, for the Live user nitrux, is nitrux. Beyond that, the operating system installer is not what you might be used to. Instead of including an “Install Nitrux” button on the Live desktop, there’s absolutely no indication as to what one should do to install the platform. Turns out, Nitrux uses Systemback for installation. Once you have the live instance up and running, click on the desktop menu and locate the Systemback entry (Figure 3).
Figure 3: Installation starts with Systemback.
The one downfall of using Systemback is it is not nearly as intuitive as many other live distro installers. Once you start the tool up, you must click System install (Figure 4).
Figure 4: Systemback is powerful, albeit not terribly user-friendly.
The first interactive section of the Systemback System installer is simple: you enter your user information. It’s the the next screen — for disk partitioning — that will trip up most users. Although Systemback will autodetect a partition, you have to delete it and create a new one, because the Mount point drop-down is greyed out. It isn’t until you delete the autodetected partition and create a new partition that you can select the mount point and file system type (Figure 5).
Figure 5: Installing Nitrux on VirtualBox with a 10GB partition.
Here’s where we have to dock Systemback another point in user-friendliness. Once you’ve selected a mount point and filesystem type, you then have to click on the left-pointing green arrow to apply the settings. Why not a simple Apply Changes button? With that complete, you’ll then be able to click the Next button, so the installation can continue and complete.
I cannot imagine how many instances of Linux I have installed over the years, dating back to the late 1990s. Even with those early iterations of Linux, I must confess, this one had me dumbfounded. No, it’s not impossible—not even slightly; but with a desktop as user-friendly as Nomad, I wonder why the developers opted to make use of a platform installer that will most likely leave new users scratching their heads and, quite possibly, giving up. That is a shame, as Nitrux is certainly something to be experienced.
A step forward
The idea of making use of snap packages is intriguing, one that would allow for:
Developers to deliver the latest version of their app
App isolation and confinement, which improves the security and reliability said app
The biggest downfall (for the moment) is that not every Linux package has been rolled into a snap. For instance, my favorite audio player, Clementine, has yet to find its way to a snap package. The über-popular Audacity audio recorder doesn’t have a snap package. The list goes on and on.
The good news is, if there’s a piece of software that doesn’t yet have a snap, it can be installed by way of the usual means (i.e., sudo apt install clementine). That means anyone using Nitrux won’t be severely limited to what software they have available to them. However, that does sort of defeat the purpose of using snap packages. When possible, at least with Nitrux, always install with snap packages.
One other feature of note is the inclusion of Android apps (Figure 6).
Figure 6: Although they’re listed in the menu, neither Anbox nor any of the installed Android apps will function.
From what it seems, users might be able to install and run Android applications. However, on two different installations, I have yet to get this feature to work. Even the pre-installed Android apps never start. What promised to be a really cool out of the box experience, fell flat.
Who is this distro for?
That’s a tough question. New users would feel right at home on the Nomad desktop — getting there, however, could be problematic. Skilled Linux users should have no problem using Nitrux and might find themselves intrigued with the snap-centric Nomad desktop. The one advantage of having a distribution centered around snap packages would be the ease with which you could quickly install and uninstall a package, without causing issues with other applications. However, that can be achieved with any distribution supporting snap packages.
In the end, Nitrux is a beautiful desktop that is incredibly efficient to use — only slightly hampered by an awkward installer and a lack of available snap packages. Give this distribution a bit of time to work out the kinks and it could become a serious contender.
Learn more about Linux through the free “Introduction to Linux” course from The Linux Foundation and edX.
JavaScript Object Notation is a schema-less, text-based representation of structured data that is based on key-value pairs and ordered lists. Although JSON is derived from JavaScript, it is supported either natively or through libraries in most major programming languages. JSON is commonly, but not exclusively, used to exchange information between web clients and web servers.
Over the last 15 years, JSON has become ubiquitous on the web. Today it is the format of choice for almost every publicly available web service, and it is frequently used for private web services as well.
The popularity of JSON has also resulted in native JSON support by many databases. Relational databases like PostgreSQL and MySQL now ship with native support for storing and querying JSON data. NoSQL databases like MongoDB and Neo4j also support JSON, though MongoDB uses a slightly modified, binary version of JSON behind the scenes.
A big part of adopting DevOps involves changing an organization’s culture. At Open Source Summit in Los Angeles, Matt Micene will host a birds of a feather session discussing how and why culture change occurs and why collaboration is the cornerstone of successfully implementing new practices.
Micene, a Senior Evangelist at Red Hat, has more than 15 years of experience in information technology, ranging from architecture and system design to data center design. He has a deep understanding of key technologies, such as containers, cloud computing, and virtualization. In this interview, Micene describes some of the challenges of DevOps adoption and the need for culture change.
Linux.com: How do you define the DevOps movement? Is it a restructuring of teams to break down silos or is it more than that?
Matt Micene:I look at it in terms of Conway’s Law — to change the way you design a system you have to change the way you communicate. Conway may have been thinking about smaller units (across teams), but I think it holds across departments and divisions. Breaking silos is about building those new bridges and paths.
You can trace the existing silos to organizational thought. “We need to control risk associated with changes, so we need a change control board. We need to limit who has access to sensitive data, so only this team has access to production systems.” With enough history, you can trace any policy to a specific event that broke production. DevOps challenges the way that organizations think about their IT processes.
DevOps methods find new ways to address the same needs; smaller changes introduce less risk than large ones, responsibilities should be on accountability not roles. Communication and collaboration are important to make sure that all of the voices involved in delivering value via an application are considered; not just in IT but also the lines of business.
Linux.com: What kind of challenges that organizations and employees generally face?
Micene: Many people look at DevOps as the next Agile or how to run an IT project. You’ll see a focus on automation and software delivery. New tools will be built or introduced, and they won’t provide the sweeping change promised.
The bigger challenges come from changing people’s points of view on where responsibilities end and how to step outside their comfort zone. “Why should a developer carry a pager? That’s why operations is here, we write code and fix bugs, they manage the application.”
And IT isn’t the sole player, getting the person who owns the bottom line to accept that 10 deployments a day to production is less risky than one big deployment a month can be difficult. Sometimes we need to change the definitions of words that we’re used to using. There’s a big difference between a “deployment” that updates a 100 lines of code to fix a feature and one that takes a whole weekend to change database schema and deploy a new version of an application that has 100s of changes and new features.
Linux.com: Does focus on culture really matter? Isn’t it all about technology?
Micene: Culture can have a lot of different definitions, especially when talking about a business. Let me offer this one without getting too academic: culture is a shared set of learned knowledge, habits, and beliefs about what is right or wrong.
The key there is that culture is shared and learned. To change culture you need to build a new consensus on what works, and then communicate it broadly. Corporate culture often isn’t planned; it’s a living aggregate of all of the experiences, and policies are the written expression.
If DevOps is about people, processes, and tools, then two-thirds of DevOps is about culture; the why and what we do things as a group. If you can’t change those, changing the how by introducing a new tool won’t get the result you want. The technology improvements are important, too. You won’t get the results you want if you’re still thinking “don’t break the nightly build” and not “the pipeline always delivers deployable code even if though build broke.”
Linux.com: A lot of companies are now consuming open source; it’s becoming a norm. Is it also creating some cultural changes?
Micene: Absolutely. Introducing new ideas into a culture sparks a reaction. Open source software makes people think differently about their IT solutions, even if you’re working with vendor-supported open source software. The rise in “inner sourcing” practices within organizations is tied to the normalizing of open source. Open source introduces the idea of collaboration to organizations that may never otherwise consider it.
Linux.com: Who should attend your session and why?
Micene: DevOps is all about people collaborating, and that’s my goal for this “birds of a feather” session. Rather than listen to someone else talk, I want to get people together to talk about their issues, ask questions, and hopefully hear some success stories too. If one person goes home with a way to un-stick a stalled effort or how to better introduce a new idea from a talk they heard at Open Source Summit, I think that’s a win.
Long Term Support (LTS) releases are as old as software. However, Canonical, the company behind Ubuntu, was the first to use the term, and now other open source projects, including the Linux kernel and many distributions, distinguish between LTS, regular, and rolling releases, each of which has different advantages and appeal to a different class of user.
Many users are content with regular releases every six to 18 months. Others who demand the very latest, prefer rolling releases, in which each package is updated whenever it is ready. By contrast, as the term implies, LTS releases are supported longer periods — typically, two to five years, although Canonical also offers Extended Security Maintenance as a paid service for another two years.
LTS releases are supported through their specified support duration by security updates, bug fixes, backports, and new device drivers, just like a regular release, although in some projects like Debian, they do not have point releases, which means that how and when fixes are applied can be different than in a regular release. Similarly, LTS releases may support only the most popular hardware architectures supported by regular releases.
Every company is now a technology company. We’re employing new digital technologies to gather data, to reach our customers, to manage the demands of a global marketplace, and to work more efficiently.
All this work requires some element of disruption. And quite often, disrupting our respective industries requires creating new systems and processes. If we acknowledge and accept that this true—and that the demands of digital transformation will only continue to increase—then we also need to spend more time and energy having organizational conversations, especially in advance of major cultural and structural changes.
As I’ve said previously: While we frequently meet to discuss topics like our market needs or leveling up our innovation methods to create new solutions, we have a tendency to delay conversations about the needs of our people, cultures, and ecosystems during digital transformation efforts.
