When Linus Torvalds first announced his new operating system, Linux, on Aug. 25, 1991, it was a “completely personal project,” Torvalds said at LinuxCon today. The kernel totaled 10,000 lines of code that would only run on the same type of hard disk Torvalds himself used because the geometry of the hard disk was hard-coded into the source code. And, he expected only other students to be interested in studying it as a theory.
Those early days were his most memorable, he said, when he was working to solve tough problems and create something out of nothing.
“Even the slightest sign of life makes you go “Wow, I really mastered this machine,” Torvalds told Dirk Hohndel, who interviewed him on stage. “You’re pumped because you got a character on the screen.”
The Linux kernel now supports more than 80 different architectures, Torvalds says, and counts 22 million lines of code with more than 5,000 developers from about 500 companies contributing, according to the latest Linux Kernel Development Report released this week. It is the big, professional project that Torvalds himself didn’t expect in that first public announcement 25 years ago.
These days, Torvalds no longer writes much code. And during the past 15 months, he was responsible for signing off on just 0.2 percent of patches submitted, according to the kernel report. Instead, he’s focused on making sure the development and release process stays on track.
“I can be proud when the release process really works and people get things done and we don’t have a lot of issues,” Torvalds.
During the past 10 years, the release schedule has stayed remarkably consistent. A new kernel is released every nine to 10 weeks, working at an average rate of 7.8 changes per hour. For the 3.19 to 4.7 releases, the kernel community added nearly 11 files and 4,600 lines of code every day, according to the report.
It has not always been smooth sailing, however. As Torvalds pointed out, “it really did take a while before it turned professional, and some of us still struggle with it at times.”
Fifteen years ago, when commercial interest in Linux began to increase but the kernel community was still very small, the process started to become unmanageable, Torvalds said. The community decided to switch to the Bitkeeper revision control system, which was a lifesaver for Torvalds “because the process before that was such a disaster,” he said.
“That was probably the only time in the history of Linux where I was like, “this is not working,” Torvalds said. “In retrospect that might have been the moment where I just gave up.”
He later created Git to further scale the development process, when Bitkeeper became too unwieldy.
Since then, things have run much more smoothly. To be sure, there have been points when Torvalds became so frustrated he considered walking away, he conceded. He would get angry and pledge to take a week off, but he would inevitably be back the next day after taking some time to cool off.
“Power management was such a bummer for so many years. We really struggled with that, where you could just take a random laptop and suspend it and resume it and assume it works,” Torvalds said.
Torvalds’ own mistakes during the 2.4 cycle also created problems with memory management that took a long time and a lot of effort to fix, he said.
For the most part, however, the technical issues have been small compared to the social challenges involved in organizing a project largely consisting of volunteers at first, and then kernel developers paid by companies with competing interests, operating in disparate markets with vastly different computing needs.
“I used to be worried about fragmentation and thought it was inevitable at some point,” Torvalds said.
This is where the GPLv2 (Gnu General Public License) license — which governs how the software can be copied, distributed, and modified — has been critical to the success of the project. The license requirement that changes to the code be made available, has been key to avoiding fragmentation that plagued other open source projects, Torvalds said. Under the GPL, developers can rest assured that their code will remain open and won’t be co-opted by corporate ownership.
“I love the GPL2,” Torvalds said. “It has been one of the defining factors of Linux.”
Today, the newest operating systems such as Zephyr and Fuchsia are being developed for tiny systems designed for the Internet of Things. Torvalds admitted that he does not look at the source code for these projects anymore. He contends that it isn’t helpful for him to look at source code for a project unless he wants to fix it. However, he stated that in order for a project to become big and attract contributors, the license is important.
“Under the GPL… nobody will take advantage of your code, it will remain free,” he said.
Red Hat updated its Kernel-based Virtual Machine (KVM)-powered virtualization platform for both Linux- and Windows-based workloads.
Red Hat Virtualization 4 includes both a high-performing hypervisor (Red Hat Virtualization Host) and a web-based virtualization resource manager (Red Hat Virtualization Manager) for management of an enterprises virtualization infrastructure. Specifically, Red Hat Virtualization 4 introduces new and enhanced capabilities around:
While virtualization remains a key underpinning for the modern datacenter, customer needs are rapidly evolving to demand more than simply virtualizing traditional workloads. Modern virtualization platforms need to address these standard scenarios while making way for the emergence of virtualized containers and cloud computing, key aspects that Red Hat Virtualization can address in an open, extensible fashion,” stated Gary Chen, research manager, Software Defined Compute, IDC.
http://www.redhat.com
The mobile gaming company must deliver a seamless experience for its gamers and allow for spikes in player activity on its Massively Multiplayer Online gaming platform. That’s why the company built a high-availability infrastructure that runs on Amazon Web Services (AWS) and allows them to launch a cluster in less than 5 minutes using Apache Mesos.
“We want to enable developers to iterate fast on their ideas and to be able to deploy new code changes as fast as possible,” say DevOps engineer Emmanuel Rieg and build and release engineer Ross McKinley, below. “We’re aiming at deploying multiple times a week, whenever a given feature is stable or bug is fixed.”
Rieg and McKinley will give a talk next week at MesosCon Europe on how they went from a blank canvas AWS account to a fully functional PaaS, to set up their immutable infrastructure. Here they give a short preview of their talk and share tips for developing on top of Mesos.
Emmanuel & Ross: We are very impressed by the diversity of services offered by Amazon. This is coupled with good AWS support in other tools we use. Developer friendliness is really important to us. The ability to run our cluster in an isolated environment (VPC) was a deciding factor.
Linux.com: How do you create high availability on the public cloud?
Emmanuel & Ross: HA is achievable on the public cloud. In our case, we couple redundancy across Availability Zone (AZ) with monitoring and autonomous systems to ensure our games can keep running. Using only one AZ will not ensure HA, as that entire zone could fail for a short time. Each of our applications runs in multiple containers at the same time. They’re are all being monitored to handle current load. When one container is down, another takes its place. The same applies for all parts of our infrastructure. All services are autoscaling and behind a service discovery system. On top of this, nodes in our cluster are deployed across multiple AZs, each of which being an isolated network with its own NAT gateway. This way we can survive a whole zone going down.
Linux.com: What role does Mesos play in your infrastructure?
Emmanuel & Ross: Mesos is the foundation we use to run all of our environments. This allows us to scale quickly, handle spikes in players gracefully, and enables our tech teams to develop with velocity.
Linux.com: Why is speed (i.e., launching a cluster in under 5 minutes) important to your business?
Emmanuel & Ross: As we use an Immutable Infrastructure, many components can be affected when performing large updates. Keeping the feedback loop short on infrastructure changes enables us to react to problems and deploy fixes with minimal user impact.
We want to enable developers to iterate fast on their ideas and to be able to deploy new code changes as fast as possible.We’re aiming at deploying multiple times a week, whenever a given feature is stable or bug is fixed. This also enables us to roll back awry deployments.
Linux.com: What is your top tip for creating development environments on top of Mesos?
Emmanuel & Ross: Have a comprehensive Monitoring solution, automate everything, and codify your infrastructure.
Good monitoring is the key to a successful development environment. Without Monitoring, you’re flying blind and will have a hard time tracking down issues.
A fully automated continuous delivery system for validating and pushing changes makes it easy to ensure that bad practices, like manual intervention and works-of-art, are avoided.
Infrastructure-as-Code is mandatory to prevent servers and infrastructure becoming a work-of-art which cannot be replicated. Treat your servers as cattle, each one is fully replaceable at any time.
Join the Apache Mesos community at MesosCon Europe on Aug. 31 – Sept. 1, 2016! Look forward to 40+ talks from users, developers and maintainers deploying Apache Mesos including Netflix, Apple, Twitter and others. Register now.
Apache, Apache Mesos, and Mesos are either registered trademarks or trademarks of the Apache Software Foundation (ASF) in the United States and/or other countries. MesosCon is run in partnership with the ASF
In Tuesday afternoon’s “Kernel Internship Report and Outreachy Panel” session at LinuxCon North America, interns and mentors involved with the Outreachy program spoke enthusiastically of their experiences with the program. The panel was moderated by Karen M. Sandler, Executive Director of the Software Freedom Conservancy, and organizer of Outreachy.
Sandler provided an overview of the Outreachy program, which offers a paid three-month internship for women and other underrepresented groups to work on a free and open source software project. Helen M Koike Fornazier, a former Outreachy intern and now a Software Engineer at Collabora, described her Linux kernel project involving video4linux, with Laurent Pinchart as her mentor. She wrote a driver, which simulates some media hardware using the Media API.
Although Fornazier’s work didn’t get merged into into main kernel, she is still developing it and hopes to get it merged later. Overall, she said, her goals within the project were met. She wrote a driver from scratch and was offered a great opportunity. “Outreachy helped a lot,” Fornazier said, noting that getting a real project to work on was key. “It’s easier than you think,” she added.
Bhaktipriya “Bhakti” Shridhar’s work, mentored by Tejun Heo, involved improving work queue implementation in the Linux kernel and removing 280 legacy workqueue interface users. Shridhar, who heard about Outreachy at school, found the Linux kernel community very supportive and expressed a wish that she could participate again.
“Having a special space for women and newbies is important. All your questions are encouraged and answered,” Shridhar said.
Outreachy allows participants to do only one internship but many go on to participate in other projects, such as the Google Summer of Code, or form new local groups, according to Sandler. “Our interns take our values and spread the ideas elsewhere,” she said.
Former Outreachy mentor Tiffany Antopolski, who is now a teacher at Mohawk College and volunteer with Kids on Computers, said her involvement with Outreachy provided momentum and helped her discover a love of teaching.
“None of this would have happened without Outreachy,” she said. “In many ways, it changed my life.”
Red Hat engineer and Outreachy mentor Rik van Riel said mentoring seemed like a good way to get more involved with the project. He said it was very satisfying to teach people and help them find the answer. He noted that communication was vital to success.
“Interns needs to ask questions,” he said. “If they are quiet, reach out to them.” The community can be intimidating, and you have to make sure participants stay engaged and keep asking questions. “Sometimes you just need to point them in the right direction,” he added.
Antopolski agreed, saying, she tells interns, “I’m not here to teach you how to code; I’m here to motivate you.” To be a mentor, she added, you have to be excited about it. “You have to love the project and you have to want to teach people.”
van Riel said participants mainly need to learn how to work within the kernel community. They already know how to program, he said, but they need to learn how to write changelogs, for example, and become familiar with the overall process.
For those considering getting involved, Shridhar’s advice is to be bold, be curious, and be open. “If you find something that you like, pursue it,” said Antopolski.
Currently, Outreachy internships are open internationally to women (cis and trans), trans men, and genderqueer people. Additionally, they are open to residents and nationals of the United States of any gender who are Black/African American, Hispanic/Latin, American Indian, Alaska Native, Native Hawaiian, or Pacific Islander. They are planning to expand the program to other participants in the future.
