Two weeks ago the Tandy Supercomputing Center in Tulsa, Oklahoma launched as the home to one of the country’s first shared, publicly available supercomputers.

The project — born of a collaboration between The University of Tulsa, Oklahoma State University, The University of Oklahoma, Tulsa Community College, the city of Tulsa, business owners and nonprofit foundations —  gives community members equal access to a $3.5 million, 100-node supercomputing system at a fraction of the cost to build their own.

Providing access to high-performance computing is also a potential boon for economic development in Tulsa by helping to speed academic research and the business community’s time to market.

“We’re lucky to live in a community in which collaboration isn’t a bad word,” said David Greer, executive director of the Oklahoma Innovation Institute. “All the university presidents came together and said, ‘We want to support it.”

A Community’s Private Cloud

Five years ago the University of Tulsa engineering department set out to build a supercomputer for the school. They quickly learned that the cost of the infrastructure to house the machines alone would eat up 60 percent of the budget. The remaining 40 percent wasn’t nearly adequate to buy the computers they needed for the job, Greer said.

Talking with other researchers in Tulsa, Greer learned it was a common problem and together they hatched a plan to address it.

“We had this naïve concept that we could pool our money and build something we could all use,” said Greer.

The Tandy Supercomputing Center is essentially a private cloud built for an entire community. The center, housed at Tulsa City Hall, holds 100 nodes with 128 GB of RAM, comprised of two 2.7 GHz Intel Xeon CPUs each running Red Hat Linux for a total of 1600 cores with about 30 Teraflops at current capacity.

But when it comes to building a collaborative supercomputer, it turns out, the technology is the easiest part.

“Politics is the biggest barrier to a collaborative model like this,” said George Louthan, director of the Tandy Supercomputing Center. “I’ve never heard of three separate universities pooling together resources to buy something none of them own.”

A Model for Collaboration

The key to securing support for the project from the business community was finding a neutral location to house the computers, provided by the city of Tulsa, which allowed all parties to come to the project on equal footing, said Greer and Louthan.

They call the collaboration model “condominium computing,” in which the center owns the entire infrastructure and members pay the operating costs based on the number of nodes, or percent compute capacity, they need. Members are then guaranteed access to that dedicated resource as well as the ability to use more resources when they’re available.

The cost for members is a one-time fee of $10,000 per node plus $2,500 per node per year in maintenance fees. But several nodes are reserved for free public use through a grant application process to startups, nonprofits and other groups that cannot afford the initial cost.

Built at one-third capacity, the center is designed to scale as the community’s computing needs grow.  Empty racks are built and ready to go with power and networking capabilities when the need arises, up to 324 nodes.

Access to Technical Support

In all, the compute resources members have access to aren’t much different from those available from any public cloud service such as Amazon’s EC2. The key advantage for businesses and researchers that buy into the Tandy center is the support they receive.

“For somebody who is familiar with scientific computing and capable of being a system administrator, it makes sense to spin up EC2 instances and then spin them down,” Louthan said.

But many of the center’s members don’t have much experience with parallel computing. And so the center has plans to employ three full-time support staff to maintain the infrastructure and work with members to plan and schedule their projects. They will also work as consultants, offering expertise and advice to businesses or researchers on how to most efficiently meet their data needs.

Center members also benefit from a water cooler effect, in which researchers and data scientists can find new ways to collaborate simply by having a common meeting place.

“It all boils down to the vision of these (university) presidents that what we have to do now as a community, as a country, is collaborate,” Louthan said. “It’s OK to compete on the athletic field; it’s not OK to compete in research and the classroom. Let’s work together.”

Embedded Linux can claim at least two great achievements in standardization in the past few years, according to Wolfgang Denk, managing director of DENX Software Engineering and creator of U-Boot, the open source universal boot loader for embedded devices. First, developers were not completely disrupted with the introduction of ARM systems.

“Thanks to Linux, the low level hardware details are well abstracted away, and on the application level it does not really matter at all which exact architecture or SoC you are working with,” Denk said via email.

And second, the rapid adoption of the Yocto Project, the open source build system, as “the best approximation of a standard Linux distribution for embedded systems we ever had so far,” he said.

In this Q&A, Denk discusses the upcoming U-Boot release; the future of U-Boot; the current state and future of embedded Linux and Android; the role of the Yocto Project in standardizing embedded development; and the best tools for embedded developers.  

U-Boot v2013.04 was released two months ago and you have another version set for release in July. What’s new?

Wolfgang Denk: I’m happy to see that the project goes on in a well- organized, predictable way.  When I had to give up the role as U-Boot maintainer in autumn last year due to personal reasons I was not sure what would happen, but fortunately Tom Rini agreed to take over, and he has been doing a marvelous job since.  He and the other custodians, and of course the huge number of developers who form this community, continue to drive this project.  It’s reassuring to see that the project has reached a point where it can stand and walk on its own feet.

The most interesting recent features are the improved support for cryptographically signed images (as used for example by the Chromium project), and the “Falcon” boot mode, which allows for very short boot times while still maintaining the flexibility of the full U-Boot toolbox.

What are important future activities in U-Boot development?

Two important tasks that are in early prototyping stages right now are adapting the Linux kernel’s Kconfig approach for configuration, and the development of a new device driver model. I highly appreciate all efforts to further these developments and expect to hear more about this at the U-Boot summit at the Embedded Linux Conference Europe in Edinburgh.

What’s the current state of embedded Linux, especially given the rise of Android and mobile devices?

Linux is – and has been for a number of years – the unchallenged top OS for all kinds of applications, including and especially for embedded applications.  But the really dramatic changes do not happen in Linux, but in the hardware.  If you consider the landslide-like move from Power Architecture to ARM systems in the last two or three years it is highly notable that this happened without disconcertment for both developers and users: thanks to Linux, the low level hardware details are well abstracted away, and on application level it does not really matter at all which exact architecture or SoC you are working with.  This is really a great achievement.

We see an increased usage of Linux in environments where security and/or reliability is essential, like thermal and hydro power plants, smart meters and such, but also in systems with hard real-time requirements, where we use the Xenomai real-time extension, which also allows to emulate proprietary RTOS like pSOS+ or VxWorks so existing code can be re-used.

With our focus on the industrial embedded market we don’t see much of Android yet.  Of course we support it, and there are a few customers who ask for it, but at least so far Android does not really play a role in the DENX business.  Most customers in the embedded market understand that Android is a good OS and software stack for smartphones and tablets, but not really well suited for classical embedded projects.  This has been discussed often enough before so I don’t have to repeat the arguments. If you are interested, read The Penguin and the Droid – a Comparison or Defining Android vs. Embedded Linux.

Where is embedded Linux headed?

With the quickly growing capabilities (CPU performance, memory and storage sizes) we see less and less need for resource-optimized configurations.  This allows a growing number of systems to use (either unchanged, or with minimal adaptations) off-the-shelf solutions like for example the Linux Foundation’s Yocto Project, or even standard distributions like Ubuntu.  This allows for standardization, and the growing size of the community leads to improved overall quality, both very important benefits.

I see more and more companies pushing their code into U-Boot and Linux mainline. This is a very good thing, but there is still a long way to go for many of them.  Especially most of the chip vendor trees are only usable for prototyping, but not for real life projects.

An interesting trend is availability of a new generation of user-customizable hardware, here by combining traditional SoCs with FPGAs on a single chip, as for example in Altera SoC or the Zynq SoC.

What’s your involvement in the Yocto Project?

We have long been waiting for a project like Yocto – a community driven open source project and with the full commitment of the Linux Foundation.  We were happy to drop efforts of developing our own “embedded distribution” and switch to basing related works on top of Yocto right from the beginning.

Where is the Yocto Project headed and what do you see as its role in embedded Linux development?

The Yocto Project has become the best approximation of a standard Linux distribution for embedded systems we ever had so far.  We see more and more customers, projects and vendors adapting this approach. This is really a great achievement.

What are some advantages of the Denx Embedded Linux Development Kit for embedded developers?

The ELDK is intended as a Yocto jump-start package.

To use Yocto on a custom board, you will have to create your own board configuration and then run a build of the tool chain (SDK) and the target packages.  This includes the download of several gigabytes of data and requires significant CPU power and storage capacity.  In short, it takes a long time before you can actually run your first example code.

ELDK allows you to download and install precompiled binaries that are not specific to just one board, but can be used for any system based on one of the supported CPU architectures.  This allows you to start developing and testing your own code within minutes, without need for lengthy and resource intensive software builds of the cross tool chain or the target libraries and tools.

But of course ELDK also allows customization for specific hardware, so the customers can use it as a software production environment for their own specific projects – a task that before Yocto everybody had to solve on his own.

What are some other essential tools for embedded developers?

First of all git; without git projects like Linux or U-Boot or so many others could not be run even nearly as efficiently as they are today, if at all.

Yocto and the underlying bitbake build tool are essential when you start building your own target images or even your own project or product-specific distribution.

The Linux kernel offers a number of powerful tracing tools which appear to be largely unknown, which is a pity.

Last but not least, people can get a lot of work done if they really know and use the standard Unix toolbox – give me bash, find, grep, sed and I will feel at home; add awk, perl, python, gcc and gdb and I will need little more.