Software Metadata Standards Wrap Up Bigger Connections

You’re in the news. But not with the headline you want.

You’re not getting attention because of your choice of text editor or the number of spaces you use to indent code blocks. However motivating those preferences are for you and me, the non-technical world sees them as private choices. You find your code in the headlines for a different and unpleasant reason: open source dependency management.

We have dependencies, of course, because we know not to “reinvent the wheel”; instead, we software experts re-use the implementations others have created. However, when done poorly, dependency management introduces more risk and degrades the quality of your application. For example, failure to comply with license requirements might be the problem.  Even worse: the absence of a license tied to a component you embedded in your application. In both cases, there are potential legal implications. 

Still more traumatic is a media headline announcing that a vulnerability just breached your organization in one of those dependencies. Projects frequently re-use software components to simplify or accelerate development; but sometimes, it can have detrimental results by introducing said vulnerabilities.

That’s not all:  suppose you are experienced and thoughtful enough to recognize this hazard and commit to good dependency management.  It turns out that’s a harder problem than might first appear, and certainly not the kind of thing that can be slipped into a project on its last days, without significant time or other costs.

Building a Standard for Software Bill of Materials

How, for instance, does an industrial oven manufacturer communicate that one of its products depends on a particular library with a known vulnerability?  How does it say that it does not have such a dependency?  One of the difficulties comes from mixing open and closed information sets. What happens in a scenario where an automotive chip uses an open source sorting algorithm, but the auto manufacturer wants to keep the use of that algorithm proprietary? 

Without a better alternative, any discussion about the algorithm has to occur under cover of a non-disclosure agreement (NDA), often one written specifically for the business and technical situation.  Where developers investigating a particular piece of software might be accustomed to connecting to GitHub and inspecting the source in question in a few seconds, even the simplest proprietary questions sometimes take months of legal, security, and compliance negotiation to begin to examine. “Manual” inspection, in any case, is unscalable.  The average application contains 200 OSS components, and each component might manually take three hours to inspect.  Does your project have a better use for 600 hours of effort?  Open source truly begins to pay off when it’s inspected not just by expert engineers but by automatic tools.

Recognize, moreover, that transitive dependencies make dependency management a harder problem than first appears.  Many of the most notorious breaches occurred not because anything was wrong with the source of a product or even the source of the libraries on which it depends; the vulnerability only turned up in a library used by those other libraries.  Over and over again, CEOs who’ve asked, “does $SOME_PROBLEM affect us?” have received the answer, “we don’t know yet: we’re not sure where it shows up in our systems.” We need transparency about dependencies and enough intelligence and standardization around hierarchical relationships to “trace the whole tree.” Organizations must track dependencies through to the operating system run-time and sometimes down to “the silicon,” that is, the microprocessor on which the software runs.

It’s a hard problem but also a solvable one.  Part of any solution is a well-defined software bill of materials (SBOM or sometimes SBoM). That’s where Kate Stewart’s career began to track this story.  Stewart currently serves the Linux Foundation as a vice president of Dependable Embedded Systems.  In previous assignments with such employers as Motorola, Freescale Semiconductor, Canonical, and Linaro, she frequently faced challenges that mixed technical and legal aspects.  As she explained her long-time focus in a recent interview, “if open source components are going to be in safety-critical places … [we need] to be able to trust open source in those spaces …” Good SBOM practices are simply necessary for the level of trust we want to have not just in industrial ovens, but airplanes, medical devices, home security systems, and much more.  An SBOM organizes such metadata about a software artifact as its identity, verification checks it hasn’t been tampered with, copyright, license, where to look up known security vulnerabilities, dependencies to check, and so on. Think of an SBOM as an ingredients list for your software.  It makes those ingredients visible, trackable, and traceable.  It lets you know if you have used the highest quality and least risky open source components to build your software.

Enter SPDX

Stewart and other technologists eventually began to team with specialists in intellectual property, product managers, and others. They developed such concepts in the early years of this millennium as SBOM, the Software Package Data Exchange (SPDX), and the OpenChain Specification.  She co-founded SPDX in 2009 to pursue “[a]n open standard for communicating software bill of materials information ….” Among other features and benefits, these frameworks provide standard and scalable ways to discuss dependencies.  

Instead of each vendor having to certify that each of its releases has been verified for security and license compliance of each of eight hundred JavaScript libraries, for example, many of the most time-consuming aspects of compliance can be automated.  When a new vulnerability is identified in an implementation of a networking protocol, automated methods can largely be applied to determine which products embed known vulnerable libraries, even while we developers remain largely unaware of the details of each component and dependency they use.  For Stewart, standards-based transparency and best practices are prerequisites for the security of safety-critical communities she helps serve.  As Stewart observes, “you can’t really be safe unless you know what you’re running.”

Daily headlines

Does that sound mundane?  The reality’s far different:  SBOM and related technologies actually play roles in events on the world stage.  For example, on the 12th of May, 2021, US President Biden issued Executive Order 14028 on Cybersecurity Improvement; SBOMs play a prominent role there.  The Open Source Initiative just named Stefano Maffulli its first Executive Director precisely because of the need for mature open source licensing practices.  Dr. Gail Murphy argued in a recent interview that it’s time to recognize that open source software is a “triumph of information-hiding [and] modularity …” in enabling the remarkable software supply chains on which we depend.  Emerging information on breaches including SolarWinds, Rapid7, Energetic Bear, and especially the latest on Juniper’s Dual-EC affair shows how disastrous it becomes when we get those supply chains wrong.  The most prominent breaches in computing history have been tied to component vulnerabilities that seemed peripheral until break-ins demonstrated their centrality.

Drone strikes?  Vaccine efficacy?  Voter fraud?  International commerce?  Nuclear proliferation?  Questions about software and data reliability and fidelity are central to all these subjects, not mere technical tangents.

That’s why SPDX’s management of hierarchical relationships is so crucial.

ISO/IEC 5926:2021 introduces SBOM standard

SPDX went live as an official international standard at the end of August.  With that milestone, standardization lowers many of the hurdles to the successful completion of an SBOM project.  Implementation becomes more consistent. “Bookkeeping” about external parts becomes largely a responsibility of the standard.  Software engineers focus more on the details specific to an application.  Then, as those external parts–the ingredients of an SBOM recipe–age and security vulnerabilities are discovered in them, developers can reliably track those components to the applications where they were used and update components to newer, hardened versions. What does that mean for you?  In your own work, the faster you identify and update vulnerable components, the less likely the chance you will have of becoming the next breach headline following an attack.

SPDX’s standardization fits in the frameworks of the International Organization for Standardization (ISO) the International Electrotechnical Commission (IEC).  ISO is a post-war transnational creation that originally focused on bolt sizes, temperature measurements, and medical supplies.  ISO tracks human affairs, of course, and its attention in recent years has shifted from materials to business processes and, in this millennium, to software.  IEC is a prior generation’s initiative to pursue the same kinds of standardization and cooperation, specifically in the realm of electrical machinery; the IEC and ISO often collaborate.

In bald terms, ISO and IEC matter to you as a programmer because governments trust them.  The new standard is sure to make its way rapidly into procurement specifications, especially for government purchases.  Suppliers become accustomed to compliance with such standards and apply them in their practices more generally.  The earlier ISO 9000 collection of standards has already greatly influenced software development.

Important though abstract

The impact and scope of ISO:IEC 5926:2021 is a challenge to understand, let alone explain.  On the one hand, millions of working programmers worldwide go about their daily chores with little thought of SPDX or even SBOMs.  While we all know we depend on packages, we largely leave it to Maven or npm, or RubyGems, etc., to handle the details for us.  Standardization of SPDX looks like a couple of layers of abstraction, even more remote from the priorities of the current sprint or customer emergency on our desks right now.

And it’s true:  SPDX is abstract, and its technical details look dry to some programmers, the opposite of the “sexy” story many start-ups aspire to.  

Without this infrastructure, though, the development of many large, complex, or mission-critical projects would grind to a halt from the friction of communication about proprietary dependencies on open source artifacts.  Think of it on a weight basis:  as the Linux Foundation’s own press release underlines, “… between eighty and ninety percent of a modern application is assembled from open source software components.” SPDX is immensely important at the same time as it’s uninteresting to all but the most specialized practitioners.

Look to history for examples of how momentous this kind of standardization is.  The US’s Progressive movement at the beginning of the twentieth century is instructive.  While often taught in ideological terms, many of its greatest achievements had to do with mundane, household matters:  does a milk bottle actually contain milk?  Can standard doses of medicines be trusted?  Is a “pound” in a butcher’s shop a full sixteen ounces?  Standards in these areas resulted in more convenience and transformed commerce to enable new market arrangements and achievements. That’s the prospect for SPDX:  more transparent and effective management of software dependencies and interactions will have far larger consequences than are first apparent.  Notice, for instance, that while the standard examples of its use have to do with open-source software, the standard itself and the tools that support it can also be applied to proprietary software and other intellectual property.  SPDX doesn’t solve all problems of communicating about dependencies; it goes a long way, though, to clarify the boundaries between technical and legal aspects.

Long lead time

The significance and need for secure software supply chains haven’t made SPDX’s adoption easy, though.  Stewart reports that individual companies drag their feet: “why should we do something before we have to?” these profit-oriented companies reasonably wonder.  Even in the best of circumstances, when an industry has largely achieved a technical consensus, “From first proposal to final publication, developing a standard usually takes about 3 years.“

Stewart herself cites this year’s Executive Order as crucial: “the one thing that made a difference” in pushing forward adoption of SPDX in 2021 was the emerging SBOM requirements that followed EO14028.  Much of her own emphasis and achievement of late has been to get decision-makers to face the reality of how crucial their dependence on open source is. No longer can they restrict focus to the 10% of a proprietary product because supply chain attacks have taught us that the 90% they re-use from the software community at large needs to be exposed and managed.

Publication of a standard mirrors application development in having so many dependencies “under the covers.” It’s not just Stewart who worked on this for more than a decade, but, as I’ll sketch in follow-ups through the next month, a whole team of organizations and individuals who each supplied a crucial requirement for completion of ISO/IEC 5926:2021.  When you or I think of great software achievements, our memories probably go to particular winning prototypes turned out over a weekend. Standards work isn’t like that.  The milestones don’t come at the rapid pace we relish. Successful standards hold out the promise, though, of impacting tens of thousands of applications at a time. That’s a multiplier and scalability that deserves more attention and understanding.  

SBOMs for everything

And that’s why ISO/IEC 5926:2021 is good news for us.  We still have licensing and security issues to track down. We still need to attend meetings on governance policies. Management of proprietary details remains delicate.  Every project and product needs its own SBOM, and vulnerabilities will continue to crop up inconveniently. With the acceptance of ISO/IEC 5926:2021, though, there’s enough standardization to implement continuous integration/continuous deployment (CI/CD) pipelines usefully. We can exchange dependency information with third parties reliably. SPDX provides a language for describing dependency management chores. SPDX gives answers that are good enough to focus most of our attention on delivering great new functionality. 

The best practices of application development applied by developers as a learned methodology can be something more than an exercise in walking a tightrope of intellectual property restrictions. Enterprise-class proposal requests become more engineering than lawyering.  You have a better shot at being in the news for your positive achievements rather than the security calamities into which you’ve stumbled.

Check in over the next several weeks to learn more about what SPDX means to your own programming, how SPDX is a model for other large-scale collaborations the Linux Foundation enables, and how teamwork is possible across profit-making boundaries.  In the meantime, celebrate ISO/IEC 5926:2021 as one more problem that each project does not have to solve for itself.

About the Author: Cameron Laird is vice president of Phaseit, Inc., where he implements software projects and publishes articles about the results. A long-time developer, manager, and author, he’s most recently concentrated on architectural challenges of “continuous everything”: continuous integration, continuous testing, and so on.