Home Blog Page 383

Civil Infrastructure Platform Takes Open Source to an Industrial Scale

One of the less discussed uses for open source software is actually in the role that it plays for industrial-scale hardware. Whereas power plants, factories, and other large infrastructure projects were once ruled over nearly entirely by operational technology (OT) control systems, in recent years, information technology — built on open source software — has been making its way onto the scene in an increasingly significant way.

Additionally, another surprising fact is that the this push to use open source in complex hardware operations has been embraced by industry leaders. One company helping to lead the charge is Siemens, one of the world’s largest producers of hardware devices, Siemens. Siemens plays an active role in advancing open source in the industrial space, with a focus on making open source security a priority for development, in part through their involvement in the Civil Infrastructure Platform (CIP) initiative.

Wolfgang Mauerer, a professor of theoretical computer science at the Technical University Regensburg, and a senior key expert at Siemens’ Corporate Research Competence Centre Embedded Linux, says that his company has “been actively supporting open source for quite a while now and actually there’s a fair amount of products that run Linux, based from Siemens.”

Siemens Depends on Open Source for Meeting Long-Term Requirements

One of the major product lines that Siemens produces is MRI scanners, which are used in hospitals to help give doctors a better picture of what is going on inside their patients. These machines, which Mauerer terms as not being traditional industrial devices but engineered in the same way, run on Linux.

“We chose Linux for these devices because we can satisfy quite different requirements that way,” explains Mauerer. “We have real-time requirements in these machines, and Linux is the only operating system that can satisfy these needs.”

Mauerer says that their decision to turn to open source was made in part because they needed an operating system that would be flexible enough to work with a wide range of needs over time.

As opposed to most consumer products which normally have a shorter lifespan of only a few years before being replaced, industrial systems are expected to last for a decade or more. As such, they need to be supported longer and have a system which can adapt with new updates as they are needed. He adds that there were concerns that commercial operating systems could become outdated over the lifetime of the devices, and that only something like Linux could give them the dependability and longevity that is required.

Mauerer points out that if they were dependent on a closed system, “then we couldn’t retrofit it with the real-time capabilities that we need.”

Siemens is currently running a number of open source projects that receive external contributions from universities and others, including their partitioning tool called the Jailhouse Hypervisor.

Securing the Future of Development for Infrastructure

Along with companies like Toshiba and Hitachi, Siemens is a founding member of CIP, which was created with the aim of “establishing an open source ‘base layer’ of industrial grade software to enable the use and implementation of software building blocks in civil infrastructure projects.”

As a member of CIP’s technological steering committee, Mauerer says that they hope to encourage a more secure environment for collaborative security. “One reason for founding CIP is that we would like to share important patches to the kernel exports,” he says, noting the lack of a central authority for ensuring that best security practices are upheld throughout their user and contributor base.   

Over the long term, Mauerer says that the goal of the CIP is to, “Really offer a set of base components from the kernel to the most important user’s base packages that we maintain over these timeframes and that all the partners, all the members, even the CI initiatives use, thus saving them effort.”

In part of their effort to establish a working base for projects that will provide users with real value, the CIP initiative has developed their own kernel for performing quality integration tests that they are calling Board at Desk, which is maintained by Ben Hutchings, who is best known for his role as the package maintainer in the Linux Debian project.

Their hope is that over time, they will establish a baseline for infrastructure related projects, ranging from rail to power plants, that developers will look to for holding to best practices.

“In the long run, we will come up with a standardized set of test and quality measures that if they’re satisfied by the kernel, will then earn them the CIP quality certification.”

Shifting the Industry Towards Open Source

One misperception Mauerer and his colleagues at CIP have to battle is the idea that companies are putting themselves at risk by working with an open source model.  

“Astonishingly, I hear these arguments that if we open source our code — if you put out anything in the public — then it becomes less secure because people can search the vulnerabilities and so on. I hear these quite frequently from medium-sized corporations and small businesses,” he says, chalking up the perception here to a “lack of expertise of dealing with the openness, so they confuse open source with a system that’s open for everyone.”

Thankfully, he is seeing far more acceptance of working with open source from the bigger players in the industry. “Most larger companies by now have realized that security by obscurity of course doesn’t work,” says Mauerer, noting that, “Giving out all the mechanisms for review by independent experts and third party experts actually improves security, making security stronger because security holes can be found proactively before they are detected out in the wild.”

For now, many companies are keeping their newfound appreciation for the power of open source as an internal secret policy, choosing not to publicize it for fear of negative pushback from the skeptics. However, Mauerer says that if you know where to look, you can see that there is real interest in pursuing greater open source usage.

“So sometimes companies are still reluctant to say in public that they are using open source. But if you go to any Linux Foundation events, you will find everyone from all industries looking outward to what has happened, placing that, of course not just because of interest but because people are using these products and these components very, very openly, very, very, very much in their products.”

The Long Road Ahead

In building their solutions, Mauerer makes a point that the base of software they create should in part be judged on whether it is sustainable over a significant amount of time. Whereas many software products, say a mobile phone’s operating system, can hold up for five years, industrial systems have life spans that can be expected to reach upwards of 25 years in some cases and face far more stress and requirements.

While he is hopeful that the benefits of more modern technologies from the IT world will filter into the industrial/infrastructure space, Mauerer also notes that industrial technology is still in its own category and that the pace of change will differ.

“We’re only very slowly picking up seconds from IT, so people who get into OT from IT will need a lot of patience at first,” he says. “But once they think the issues through, they will realize that this pace is something that’s vital to the industry because just imagine a power station that’s programmed in the same way as a mobile phone and has as many software problems as mobile phones. Obviously you want to avoid having a power outage twice or three times a day.”

Keeping their environment secure takes a considerable effort, one that is not always recognized for the commitment that it takes to keep things running smoothly. Mauerer quips that, “it’s work that we need to do, that must be done but we don’t get any points for doing it. We just lost points for not doing it.”

For more information about Civil Infrastructure Platform, visit https://www.cip-project.org/.

LC3 Schedule Announced, Register Now | LinuxCon + ContainerCon + CloudOpen 中国论坛日程表现已公布,立即注册

Join us in Beijing June 25 – 27, for three days of education across 175+ sessions, collaboration opportunities with open source technologists and professionals from around the globe, and the chance to learn about the newest trends and topics in open source.

Keynote speakers include:

  • Kelsey Hightower, Developer Advocate, Google
  • Abby Kearns, Executive Director, Cloud Foundry Foundation
  • Greg Kroah-Hartman, Linux Kernel Maintainer
  • Michelle Noorali, Senior Software Engineer, Microsoft
  • Linus Torvalds, Creator of Linux & Git, in conversation with Dirk Hohndel, VP & Chief Open Source Officer, VMware

Read more at The Linux Foundation

Quantum Mechanics Could Solve Cryptography’s Random Number Problem

Peter Bierhorst’s machine is no pinnacle of design. Nestled in the Rocky Mountains inside a facility for the National Institute of Standards and Technology, the photon-generating behemoth spans an entire building. Its lasers, mirrors, and lenses are split among three laboratories, two of them at opposite ends of the L-shaped building. The whole thing is strung together with almost 900 feet of optical fiber. “It’s a prototype system,” the mathematician explains. “Something might drift out of alignment, and the whole thing stops working. It might take a few days to figure out what went wrong.”

On a good day, the machine produces 1,024 bits of data every 10 minutes, equivalent to typing 13 letters per minute. But it promises what even monkeys on typewriters can’t: completely random text.

It’s like this: Even if you repeat a quantum experiment by preparing a quantum particle in exactly the same initial state, subjecting it to the exact same conditions, measuring its orientation after the same amount of time, you can still end up with totally different results. This is unlike flipping a quarter, where its initial conditions—the force of your thumb, the direction of the winds—determine the outcome before it lands. The outcome of “flipping” a tiny quantum particle only exists as probabilities until the moment it “lands.” Electrons, photons, and atoms are really, actually random.

Read more at Wired

DevOps Is the Secret Ingredient to Make Microservices Cook

The skill in DevOps is not being a great chef, but a great manager: Managing the waiters, the hot window, the prep chefs, and the money, all from a vantage point above the floor, with full visibility of the entire chain of processes, products, and people.

In the microservices world, this means it’s generally DevOps’ duty to set up all of the infrastructure required to build out at-scale environments. That means Web application servers, registries and repositories, OS and container images, virtualized networking, firewalls, load balancers, message queues, and reverse proxies. It’s also up to the DevOps team to support new technologies demanded by the development teams: HTTP2GRPC, and reliable SSL. …

Today, we know that stateful and stateless applications can both happily coexist in the cloud, but the actual day-to-day work of managing that data isn’t always easy. Georgi Matev, head of product at Kasten, said that, “What we are seeing is that data is following the same pattern as we’ve seen on the compute side. As things break into smaller and more logically sized components, the same makes sense on the data side.”

Read more at The New Stack

Keep Database Deploys Separate

A while ago, I tweeted that you shouldn’t deploy database migrations at the same time as your app code. I thought I’d write something about why I feel this way, and the situations where I feel strongest about this.

This post has the structure of an Architecture Decision Record, because context matters. I think ADRs are an excellent way to force you to think about the context in which you make a decision, and I’m hoping to use the format to think about the context in which my recommendation applies.

The team that this recommendation applies most to has the following things going on:

  1. You are building a user-facing application, quite probably a web app for consumption by a browser or mobile native application.
  2. You have enough users that, whenever you deploy, most likely someone is using the app. Therefore, you employ techniques to deploy your app with zero downtime, such as deploying one app server at a time rather than all at once.
  3. You deploy your app multiple times a week (possibly even multiple times a day).
  4. You don’t have maintenance windows (for deployments or otherwise).

Read more at PhilandStuff

Tig – A Command Line Browser for Git Repositories

In a recent article, we’ve described how to install and use GRV tool for viewing Git repositories in Linux terminal. In this article, we would like to introduce to you another useful command-line based interface to git called Tig.

Tig is a free open source, cross platform ncurses-based text-mode interface for git. It is a straight-forward interface to git that can help in staging changes for commit at chunk level and works as a pager for output from different Git commands. It can run on Linux, MacOSX as well as Windows systems.

Read more at Tecmint

 

How to Connect Wi-Fi from Linux Terminal Using Nmcli Command

There are several command-line tools for managing a wireless network interface in Linux systems. A number of these can be used to simply view the wireless network interface status (whether it is up or down, or if it is connected to any network), such as iwiwlistipifconfig and others.

And some are used to connect to a wireless network, and these include: nmcli, is a command-line tool used to create, show, edit, delete, enable, and disable network connections, as well as control and display network device status.

First start by checking the name of your network device using the following command. From the output of this command, the device name/interface is wlp1s0 as shown.

Read more at Tecmint

Real-World Build Tips for Yocto

Despite its widespread and growing adoption, the Yocto Project is one of the more misunderstood of Linux technologies. It’s not a distribution but rather a collection of open source templates, tools, and methods for creating custom embedded Linux-based systems. Yocto Project contributor and Intel Embedded Software Engineer Stephano Cetola explained more about Yocto in his talk at the recent Embedded Linux conference in Portland.

Although embedded hardware vendors often list “Yocto” along with Ubuntu, Fedora, and the like, one Yocto Project build is often markedly different from another. Embedded developers who once constructed their own DIY Linux stacks from scratch to provide stripped-down stacks optimized for power savings and specific technologies now typically use Yocto. In the process, they save countless hours of debugging and testing.

Yocto is a line drawn in the sand — we’re picking certain versions of software, drawing a line and testing them,” Cetola said.  “Every day we test four different architectures and build on six different distros checking for compatibility and performance. We can provide you with BeagleBone SDKs and images, and there’s a bug tracker too.”

Most of this was old news to the attendees at Cetola’s talk, “Real-World Yocto: Getting the Most out of Your Build System.” Yet, there’s a lot about Yocto that even experienced developers don’t know, said Cetola. His talk covered a variety of Yocto tips, with a focus on using the BitBake build engine.

Cetola highlighted some of his favorite best practices, utilities, scripts, and commands, including wic (OpenEmbedded Image Creator), Shared State cache, and packaged feeds. (You can watch the entire presentation below.)

Layering up

The Yocto Project is built around the concept of layers. This is often difficult for developers to grasp so they often just ignore it. “People tend to lump everything into one giant layer because it’s a relatively quick way to get your build started,” said Cetola. “But if you put all your distro information, hardware requirements, and software into a single layer, you’ll be kicking yourself later.”

The biggest drawback is the difficulty in updating hardware and software. “When your boss comes in and drops a web kiosk project on your desk and asks if this will work on your layer scheme, the only way you can answer yes is if you have separated out the layers.”

Having a separate distro layer, for example, makes it easier to support both a frame buffer and an X11 layer, says Cetola. “If your hardware uses different architectures or you don’t want your different hardware mingling together you can separate them into layers to help you distribute these internally,” he added. “In software, you may have Python living with C, and if they have nothing do with each other, separating them means you can ship the manufacturer or QA team only what they need.”

Wic’ing your way to bootable image formats

Yocto Project developers often struggle when integrating a vendor layer into a bootable image, said Cetola. “In the past it’s been hard to add multiple partitions or try to do a layered architecture where your layering read-only and read-write on top of each other.”

A new tool called wic (OpenEmbedded Image Creator) can help out. Wic reads from a kickstart WKS file that allows you to generate custom partitions and media that you can burn to,” explained Cetola. “For example, your manufacturer may expect to get an SD card, but you may also want to boot from NAND or NOR. Wic lets you cleanly separate these concepts, and then reuse them.”

Wic lets you copy files into or remove files from a Wic image, as well as ls a directory inside an image for greater introspection. Wic also supports bmap-tools, “which is an order of magnitude faster than using dd,” said Cetola. “Bmap-tools realizes that you’re going to copy useless data, so it skips that by doing a sparse copy. Once you use bmap you’ll never use dd again.”

SSTATE is your friend

One of the blessings (in flexibility and code purity) and curses (drudgery) of Yocto development is that everything is built from scratch. That means it must be rebuilt from scratch over and over again. Fortunately, the platform offers a workaround in the form of shared state cache (SSTATE), which is sort of a snapshot of an unaltered Yocto recipe deployed as a set of packaged data generating a cache. BitBake checks SSTATE to see if a recipe does not need to be rebuilt, thereby saving time.

“One of the biggest complaints about Yocto is that it takes a long time to build, and it does — Buildroot wins on speed every time,” says Cetola. “SSTATE cache, which is meant to be used circumstantially, can speed the process up, but a lot of people don’t take advantage of it.” Cetola said there were numerous examples in which building from scratch “isn’t ideal,” for example when a team is doing a build on underpowered laptops.

Cetola recommended using site.conf, which he described as “a configuration file that BitBake looks for when it starts up inside the conf directory.” He continued: “I have a script that starts my build directory and copies in my site.conf, which sets the download and SSTATE directories.”

Cetola also suggested that developers make greater use of the related SSTATE mirror technology, which he said is “handy for sharing with other machines.” These can also be managed by site.conf. “People think they won’t benefit from SSTATE mirrors, but it’s extremely useful.”

Package feeds

Another way to accelerate the build process is to use package feeds, which “can not only save an immense amount of time during your personal development, but also for anybody who needs to quickly install the software,” said Cetola. He described a scenario in which “you use dd to burn something to a card, and you load it on a board and it doesn’t work. So you change your software, boot it again, burn it, load it, and it’s missing a library.”

The missing library is “probably sitting in an rpm folder,” said Cetola. “Yocto can run a BitBake package index, which indexes the folder so rpm can look for it, so you essentially have a repo. By creating a package feed, and sharing that folder on a webserver, and running BitBake package index, you’ve saved yourself the trouble of pulling the SD card and copying something onto it. Instead you just say ‘rpm install’.”

Yocto grows introspective

Sometimes the problem is not a missing file, but rather an unexpected one. “Customers always say to me: ‘I just built my file system and booted my board, and there’s a file there — where did it come from and why is it there?’”

To figure out what the hell is going on — a process known as introspection — Cetola starts with oe-pkgdata-util to find the path. “It should output the name of the recipe that caused that file to populate on the board,” he explained. “You can point it any file and it will do its best to introspect the file and figure it out.” If that’s not enough, he turns to git grep, as well as DNF, “which gives you a lot of introspection onto what’s on your board and why.”

Weird files show up from time to time because “when we do a build and run a file system a lot of stuff is done dynamically,” explained Cetola. In this case, the above tools aren’t likely to help. For example, “inside the guts of Yocto there are rootfs post process commands that can slip something on to the board. For that I use IRC. One of the things I love about the Yocto Project is that it’s very friendly. The IRC channel is a great place to ask questions. People respond.”

Other tools include the “recipetool” and its appendfile sub-command, which “will generate the recipe for you” if you need to change a file, said Cetola. Developers can also use a dependency tree, which can be generated within BitBake using the -g option. “Yocto 2.5 will have an oe-depends-dot tool, which will save you from having to look at that gigantic dependency tree by letting you introspect specific parts.”

Cetola is surprised that developers don’t make greater use of BitBake options. “Whenever I go out to lunch and am running a substantial build, I use the -k option, which keeps the build from stopping when it encounters an error.” Other options include the -e option, which outputs the BitBake environment, and -C command for invalidating a stamp (specific clean).

BitBake scripts

Cetola also gave a shout out to some of his favorite BitBake scripts, starting with devtool, which the Yocto Project wiki describes as a way “to ‘mix’ customization into” a Yocto image. “If you’re not a full-time kernel developer but you need to do some edits on the kernel, devtool can be a lifesaver,” said Cetola. “Using `devtool modify` and `devtool build` you can modify and build the kernel without building an SDK or rolling your own cross compilation environment. It’s also a handy tool for generating recipes. Once you’ve finished making a small edit to the kernel, `devtool finish` can make the patch for you.”

Another useful script is BitBake-layers, “which is great when you’re building layers or searching for them.” Cetola also recommended bitbake-dumpsig/diffsings. “Say you changed one thing in your recipe and BitBake recompiled 25 things,” he said. “What happened was that the change caused different SSTATE hashes to invalidate. To find out why, you can use bitbake-dumpsig/diffsings. Dumpsigs will dump all the information (stored in the ‘stamps’ directory) into a format where you can see the things it is basing its hash on, and then use the diff tool to compare them to work out whether there was a dependency change.”

Cetola concluded with a call for community involvement. “If you’re brave enough to look at Bugzilla please do, but if that’s a bit much, just find a part of the system you’re interested in working on and send us an email — we’re always willing to take contributions and willing to help.”

Watch the entire presentation below:

Changing Healthcare with Blockchain Technology

Blockchain technology is heralded to become a broadly disruptive force in the coming years. According to a Forbes story, blockchain is already revolutionizing contracts, payment processing, asset protection, and supply chain management. However, partly due to the industry’s emphasis on records, authentication and people-centric processes, healthcare is predicted to be one of the fields that blockchain will truly transform.

That was the key message at an Open Source Leadership Summit keynote address titled “Blockchain Technology at Change Healthcare” by Aaron Symanski, CTO at Change Healthcare. In his talk, Symanski said that blockchain is already impacting the healthcare system.

Read more at The Linux Foundation

DevOps, Machine Learning Dominate Technology Opportunities This Year

Latest Stack Overflow survey of 100,000-plus developers finds the highest salaries and interest levels in DevOps methodologies and artificial intelligence.

This is a key takeaway from the latest surveyof more than 100,000 developers worldwide, conducted by Stack Overflow. The survey finds DevOps specialists and engineering managers have the highest salaries in the field, averaging between $70,000 and $90,000 a year worldwide. (Within the United States, salaries for these two range between $110,000 annually for DevOps specialists and $137,000 for engineering managers.)

The survey also shows that DevOps specialists and developers who code for desktop and enterprise applications have the most experience, averaging eight years of professional coding experience

Read more at ZDNet