Organizations looking to better locate, understand, manage and gain value from their data have a new industry standard to leverage. ODPi, a nonprofit Linux Foundation organization focused upon accelerating the open ecosystem of big data solutions, recently announced ODPi Egeria, a new project that supports the free flow of metadata between different technologies and vendor offerings.
Recent data privacy regulations such as GDPR have brought data governance and security concerns to the forefront for enterprises, driving the need for a standard to ensure that data providence and management is clear and consistent—supporting the free flow of metadata between different technologies and vendor offerings. Egeria enables this, as the only open source driven solution designed to set a standard for leveraging metadata in line of business applications, and enabling metadata repositories to federate across the enterprise.
The first release of Egeria focuses on creating a single virtual view of metadata. It can federate queries across different metadata repositories and has the ability to synchronize metadata between different repositories. The synchronization protocol controls what is shared, with which repositories and ensures that updates to metadata can be made with integrity.
Most Linux users have heard about the open source RISC-V ISA and its potential to challenge proprietary Arm and Intel architectures. Most are probably aware that some RISC-V based CPUs, such as SiFive’s 64-bit Freedom U540 found on its HiFive Unleashed board, are designed to run Linux. What may come as a surprise, however, is how quickly Linux support for RISC-V is evolving.
“This is a good time to port Linux applications to RISC-V,” said Comcast’s Khem Raj at an Embedded Linux Conference Europe presentation last month. “You’ve got everything you need. Most of the software is upstream so you don’t need forks,” he said.
By adopting an upstream first policy, the RISC-V Foundation is accelerating Linux-on-RISC-V development both now and in the future. Early upstreaming helps avoid forked code that needs to be sorted out later. Raj offered specifics on different levels of RISC-V support from the Linux kernel to major Linux distributions, as well as related software from Glibc to U-Boot (see farther below).
The road to RISC-V Linux has also been further accelerated thanks to the enthusiasm of the Linux open source community. Penguinistas see the open source computing architecture as a continuation of the mission of Linux and other open source projects. Since IoT is an early RISC-V target, the interest is particularly keen in the open source Linux SBC community. The open hardware movement recently expanded to desktop PCs with System76’s Ubuntu-driven Thelio system.
Processors remain the biggest exceptions to open hardware. RISC-V is a step in the right direction for CPUs, but RISC-V lacks a spec for graphics, which with the rise of machine vision and edge AI and multimedia applications, is becoming increasingly important in embedded. There’s progress on this front as well, with an emerging project to create an open RISC-V based GPU called Libre RISC-V. More details can be found in this Phoronix story.
SiFive launches new Linux-driven U74 core designs
RISC-V is also seeing new developments on the CPU front. Last week, SiFive, which is closely associated with the UC Berkeley team that developed the architecture, announced a second gen RISC-V CPU core designs called IP 7 Series. IP 7 features the Linux-friendly U74 and U74-MC chips. These quad-core, Cortex-A55 like processors, which should appear in SoCs in 2019, are faster and more power efficient than the U540.
The new U74 chips will support future, up to octa-core, SoC designs that mix and match the U74 cores with its new next-gen MCU chips: the Cortex-M7 like E76 and Cortex-R8 like S76. The U74-MC model even features its own built-in S7 MCU for real-time processing.
Although much of the early RISC-V business has been focused on MCUs, SiFive is not alone in building Linux-driven RISC-V designs. Earlier this summer a Shakti Project backed by the Indian government demonstrated Linux booting on a homegrown 400MHz Shakti RISC-V processor.
A snapshot of Linux support for RISC-V
In his ELC presentation, called “Embedded Linux on RISC-V Architecture — Status Report,” Raj, who is an active contributor to RISC-V, as well as the OpenEmbedded and Yocto projects, revealed the latest updates for RISC-V support in the Linux kernel and related software. The report has a rather short shelf life, admitted Raj: “The software is developing very fast so what I say today may be obsolete tomorrow — we’ve already seen a lot of basic tool, compilers, and toolchain support landing upstream.”
Raj started with a brief overview of RISC-V, explaining how it supports 32-, 64-, and even future 128-bit instruction sets. Attached to these versions are extensions such as integer multiply/divide, atomic memory access, floating point single and double precision, and compressed.
The initial Linux kernel support adopts the most commonly used profile for Linux: RV64GC (LP64 ABI). The G and the C at the end of the RV64 name stand for general-purpose and compressed, respectively.
The Linux kernel has had a stable ABI (application binary interface) in upstream Linux since release 4.15. According to Raj, the recent 4.19 release added QEMU virt board drivers “thanks to major contributions from UC Berkeley, SiFive, and Andes Technology.”
You can now run many other Linux-related components on a SiFive U540 chip, including binutils 2.28, gcc 7.0, glibc 2.27 and 2.28 (32-bit), and newlib 3.0 (for bare metal bootstrapping). For the moment, gdb 8.2 is available only for bare-metal development.
In terms of bootloaders, Coreboot offered early support, and U-Boot 2018.11 recently added RISC-V virt board support upstream. PK/BBL is now upstream on the RISC-V GitHub page.
OpenEmbedded/Yocto Project OE/Yocto was the first official Linux development platform port, with core support upstreamed with the 2.5 release. Among full-fledged Linux distributions, Fedora is the farthest along. Fedora, which has done a lot of the “initial heavy lifting,” finished its bootstrap back in March, said Raj. In addition, its “Koji build farm is turning out RISC-V RPMs like any other architecture,” he added. Fedora 29 (Rawhide) offers specific support for the RISC-V version of QEMU.
Debian still lacks toolchain for cross-build development on RISC-V, but it’s already possible, said Raj. Buildroot now has a 64-bit RISC-V port and a 32-bit port was recently submitted.
Raj went on to detail RISC-V porting progress for the LLVM compiler and the Musl C library. Farther behind, but in full swing, are ports for OpenOCD UEFI, Grub, V8, Node.js, Rust, and Golang, among others. For the latest details, see the RISC-V software status page, as well as other URLs displayed toward the end of Raj’s ELC video below.
Addressing the rapidly growing user base around GraphQL, The Linux Foundation has launched the GraphQL Foundation to build a vendor-neutral community around the query language for APIs (application programming interfaces).
“Through the formation of the GraphQL Foundation, I hope to see GraphQL become industry standard by encouraging contributions from a broader group and creating a shared investment in vendor-neutral events, documentation, tools, and support,” said Lee Byron, co-creator of GraphQL, in a statement.
“GraphQL has redefined how developers work with APIs and client-server interactions,” said Chris Aniszczyk, Linux Foundation vice president of developer relations…
Maybe you’ve heard of Go. It was first introduced in 2009, but like any new programming language, it took a while for it to mature and stabilize to the point where it became useful for production applications. Nowadays, Go is a well-established language that is used for network and database programming, web development, and writing DevOps tools. It was used to write Docker, Kubernetes, Terraform and Ethereum. Go is accelerating in popularity, with adoption increasing by 76% in 2017, and now there are Go user groups and Go conferences. Whether you want to add to your professional skills, or are just interested in learning a new programming language, you may want to check it out.
Why Go?
Go was created at Google by a team of three programmers: Robert Griesemer, Rob Pike, and Ken Thompson. The team decided to create Go because they were frustrated with C++ and Java, which over the years had become cumbersome and clumsy to work with. They wanted to bring enjoyment and productivity back to programming.
…The idea of Go’s design is to have the best parts of many languages. At first, Go looks a lot like a hybrid of C and Pascal (both of which are successors to Algol 60), but looking closer, you will find ideas taken from many other languages as well.
Go is designed to be a simple compiled language that is easy to use, while allowing concisely-written programs that run efficiently. Go lacks extraneous features, so it’s easy to program fluently, without needing to refer to language documentation while programming. Programming in Go is fast, fun, and productive.
DevOps and cloud computing have become two of the ways companies can achieve this needed transformation, though the relationship between the two is not easily reconciled—DevOps is about the process and process improvement, while cloud computing is about technology and services. It’s important to understand how the cloud and DevOps work together to help businesses achieve their transformation goals.
Different organizations outline DevOps in different ways. This article does not debate which definition is correct, but rather presents them both to focus on the cloud’s benefit to DevOps. That said, DevOps definitions generally fall into two terms:
In organizations it is defined as developer-friendly operations—IT operations are run separately yet in a way that is much more friendly to developers (e.g., self-service catalogs are provided to developers for stipulating infrastructure or providing technology-enabled pipelines for deploying new code).
DevOps as a single consolidated team is habituated in organizations—developers take on operations responsibilities and vice versa.
Companies that focus on developers for operations often use cloud computing to speed developer productivity and efficiency. Cloud computing permits developers more control over their own components, resulting in smaller wait times. This application-specific architecture makes it easy for developers to own more components. By using cloud tools and services to automate the process of building, managing and provisioning through the code, service teams speed up the development process, eliminate possible human error and establish repeatability.
There are times when you want to browse the Internet privately, access geo-restricted content or bypass any intermediate firewalls your network might be enforcing.
One option is to use a VPN, but that requires installing a client software on your machine and setting up your own VPN server or subscribing to a VPN service.
The simpler alternative is to route your local network traffic with an encrypted SOCKS proxy tunnel. This way, all your applications using the proxy will connect to the SSH server and the server will forward all the traffic to its actual destination. Your ISP (internet service provider) and other third parties will not be able to inspect your traffic and block your access to websites.
This tutorial will walk you through the process of creating an encrypted SSH tunnel and configuring Firefox and Google Chrome web browsers to use SOCKS proxy.
Recent industry events have underscored the strength of open source in today’s computing landscape. With billions of dollars being spent, the power of open source development, collaboration, and organization seems unstoppable.
Toward that end, we recently provided an array of articles, videos, and other resources to meet you where you are on your open source journey and help you master the basics, improve your skills, or explore the broader ecosystem. Let’s take a look.
To start, we provided some Linux basics in our two-part series exploring Linux links:
Understanding Linux Links: Part 1 By Paul Brown – This beginning tutorial provides an overview of hard links, soft links, and when and how to use them.
Understanding Linux Links: Part 2 By Paul Brown – This tutorial follows up with a look at some linking quirks you should know about.
Then, we covered some basic tools for open source logging and monitoring:
Open Source Logging Tools for Linux By Jack Wallen – Here, Jack provides an overview of some tools to help you mine the depths of your log files.
We also took an in-depth look at the Introduction to Open Source, Git, and Linux training course from The Linux Foundation. This course presents a comprehensive learning path focused on development, Linux systems, and the Git revision control system. The $299 course is self-paced and comes with extensive and easily referenced learning materials. Get a preview of the course curriculum in this four-part series by Sam Dean:
As the default compiler for the Linux kernel, the GNU Compiler Collection (GCC) delivers trusted, stable performance along with the additional extensions needed to correctly build the kernel. We took a closer look at this vital tool in this whitepaper:
Security is another vital component of Linux. In this video interview, Linux kernel maintainer Greg Kroah-Hartman provides a glimpse into how the kernel community deals with vulnerabilities.
Along with all these articles, we also recently published videos from some of our October events. Follow the links below to watch complete keynote and technical session presentations from Open Source Summit, Linux Security Summit, and Open FinTech Forum.
Check out 90+ sessions from Open Source Summit Europe & ELC + OpenIoT Summit Europe.
These 21 videos from Linux Security Summit Europe provide an overview of recent kernel development.
The 9 keynote videos from Open FinTech Forum cover cutting-edge open source technologies including AI, blockchain, and Kubernetes.
Stay tuned for more event coverage and essential open source resources.
Logical Volume Manager (LVM) is a software-based RAID-like system that lets you create “pools” of storage and add hard drive space to those pools as needed. There are lots of reasons to use it, especially in a data center or any place where storage requirements change over time. Many Linux distributions use it by default for desktop installations, though, because users find the flexibility convenient and there are some built-in encryption features that the LVM structure simplifies.
However, if you aren’t used to seeing an LVM volume when booting off of a Live CD for data rescue or migration purposes, LVM can be confusing because the mountcommand can’t mount LVM volumes. For that, you need LVM tools installed. The chances are great that your distribution has LVM utils available—if they aren’t already installed.
This tutorial explains how to create and deal with LVM volumes.
With Linux kernel 4.19 hitting the streets, a new version of the GNU Linux-libre kernel is now available, version 4.19, based on the upstream kernel but without any proprietary drivers.
Based on the recently released Linux 4.19 kernel series, the GNU Linux-libre 4.17-gnu kernel borrows all the new features, including the experimental EROFS (Enhanced Read-Only File System) file system, initial support for the Wi-Fi 6 (802.11ax) wireless protocol, and mitigations for the L1FT and SpectreRSB security flaws.
While the GNU Linux-Libre 4.19 kernel comes with all these goodies found in the upstream Linux 4.19 kernel, it doesn’t ship with proprietary code. Deblobbed drivers include Aspeed ColdFire FSI Master, MT76x0U and MT76x2U Wi-Fi, MTk Bluetooth UART, as well as Keystone and Qualcomm Hexagon Remoteproc.
There is a lot of activity and investment in the U.S supercomputer space now. After years of falling behind China for supercomputer superiority, the U.S is now storming back.
On Oct. 26, the Department of Energy’s National Nuclear Security Administration (NNSA) Lawrence Livermore National Laboratory (LLNL) formally dedicated the Sierra system, which is now the world’s third fastest supercomputer.
The Sierra system is built with the same basic design as the IBM Summit system, which became the world’s most powerful supercomputer in June. Summit is a 200 petaflop system, while the newly dedicated Sierra currently tops out at 125 petaflops.
Sierra is a massive system spread out across 7,000 square feet of data center space. The system has 240 computing racks and 4,320 nodes.
