LinkedIn Announces Open Source Tool to Keep Kafka Clusters Running

By

-

August 29, 2017

Today at The Kafka Summit in San Francisco, LinkedIn announced a new load balancing tool called Cruise Control, which has been developed to help keep Kafka clusters up and running.

The company developed Kafka, an open source message streaming tool to help make it easier to move massive amounts of data around a network from application to application. It has become so essential today that LinkedIn has dedicated 1800 servers moving over 2 trillion transactions per day through Kafka, Jiangjie Qin, lead software engineer on the Cruise Control project told TechCrunch.

With that kind of volume, keeping the Kafka clusters running has become mission-critical, so earlier this year the team decided to create a tool that would recognize when a cluster was going to break.

Distributed Systems Are Hard

By

The New Stack

-

August 29, 2017

In this post, we’ll look at some of the ways distributed systems can trip you up and some of the ways that folks are handling those obstacles.

Forget Conway’s Law, distributed systems at scale follow Murphy’s Law: “Anything that can go wrong, will go wrong.”

At scale, statistics are not your friend. The more instances of anything you have, the higher the likelihood one or more of them will break. Probably at the same time.

Services will fall over before they’ve received your message, while they’re processing your message or after they’ve processed it but before they’ve told you they have. The network will lose packets, disks will fail, virtual machines will unexpectedly terminate.

There are things a monolithic architecture guarantees that are no longer true when we’ve distributed our system.

How to Use the motd File to Get Linux Users to Pay Attention

By

Network World

-

August 29, 2017

It seems only decades ago that I was commonly sending out notices to my users by editing the /etc/motd file on the servers I managed. I would tell them about planned outages, system upgrades, new tools, and who would be covering for me during my very rare vacations. Somewhere along the stretch of time since, message of the day files seem to have faded from common usage – maybe overwhelmed by an excess of system messages, emailed alerts, texts, and other notices that have taken over, the /etc/motd file has. Or maybe not.

The truth is that the /etc/motd file on quite a number of Linux systems has simply become part of a larger configuration of messages that are fed to users when they log in. And, even if your /etc/motd file is empty or doesn’t exist at all, login messages are being delivered when someone logs into a server via a terminal window – and you have more control over what those messages are telling your users than you might realize.

Jump-Start Your Career with Open Source Skills

By

OpenSource.com

-

August 29, 2017

Although attending college is not required for success in software development, college programs can provide a great deal of useful information in a relatively short period of time. More importantly, they are designed to cover all necessary concepts without the knowledge holes some self-taught practitioners suffer. College programs also often include theory and history, which can form the foundation for professional exploration and decision-making.

Yet college graduates entering the workforce often find their coursework has emphasized theory over the practice, technologies, and trends required for success on the job. The reason? Curricula take time to develop, so institutions of higher education often teach technologies and practices that are at the tail end of current usage.

Fortunately, there are ways to learn and develop the knowledge and skills you need to land a job and succeed in today’s workplace.

My Use-Case for Go

By

Dev.to

-

August 29, 2017

The lack of generics is often mentioned in discussions regarding Go.

I would have liked Go to have algebraic data types and immutability by default. I would gladly give nil away to get these features.

On the positive side, Go has good libraries, good tooling, a common style and a syntax that is extremely easy to pick up. It’s fast enough and it has good support for concurrency via goroutines. It also produces executables that are very easy to deploy anywhere.

Given this description, it seems to me that Go is an evolution of C and Python and I decided to give it a try rewriting a project originally written in Python I am working on.

What Do the Most Successful Open Source Projects Have In Common?

By

The Linux Foundation

-

August 28, 2017

Thriving open source projects have many users, and the most active have thousands of authors contributing. There are now more than 60 million open source repositories, but the vast majority are just a public workspace for a single individual. What differentiates the most successful open source projects? One commonality is that most of them are backed by either one company or a group of companies collaborating together.

So, tracking the projects with the highest developer velocity can help illuminate promising areas in which to get involved, and what are likely to be the successful platforms over the next several years.

Rather than debate whether to measure high-velocity projects via commits, authors, or comments and pull requests, we use a bubble chart to show all 3 axes of data, and plot on a log-log chart to show the data across large scales.

Future Proof Your SysAdmin Career: Embracing DevOps

By

Sam Dean

-

August 28, 2017

Sysadmins are increasingly looking to expand their skillsets and carve out new opportunities. With that in mind, many sysadmins are looking to the world of DevOps. At lots of organizations, DevOps has emerged as the most effective method for application delivery, including in the cloud.

One of the drivers of the DevOps movement is that organizations simply have limits on the number of IT staffers, sysadmins, and developers that they can employ. Cross-pollination of traditional skillsets makes good business sense. And, as Jeff Cogswell has noted, “The line between hardware and software is more blurry than it used to be.”

Cogswell also laid out a good recipe for what specific skills to master in order to meet DevOps goals:

Learn what virtualization is and how, through software alone, you can provision a virtual computer and install an operating system and a software stack.
Study emerging open source platforms and frameworks, such as OpenStack.
Learn network virtualization.
Learn to use configuration management tools, such as Puppet and Chef.

All of these pursuits can help sysadmins appeal to organizations looking to create more collaborative and efficient working environments. Additionally, as mentioned earlier, fluency and facility with emerging cloud, virtualization, and configuration management tools can make a substantial compensation difference for sysadmins.

Training options

Sysadmins interested in becoming more fluent with DevOps skills and practices can start by exploring Dice’s Skills Center. A look there makes it clear that skillsets surrounding configuration management tools, containers, and open platforms are much in demand. Savvy sysadmins can combine existing competencies with these skillsets and move the needle.

Flexible training options are available for these tools. For example, if you just want to take Puppet for a test drive within a virtual machine, you can do so here, or there are instructor-led and online training options detailed on the same page. For example, you can chart a learning roadmap for Puppet, find in-person or online training options for Chef, or sample some of the available online tutorials.

A great way to learn more about cloud skills is to open an account on Amazon Web Services and work with EC2 technology. OpenStack training options also abound. The Linux Foundation, for example, offers an OpenStack Administration Fundamentals course, which serves as preparation for certification. The course is available bundled with the COA exam, enabling students to learn the skills they need to work as an OpenStack-skilled administrator and get the certification to prove it.

The Guide to the Open Cloud 2016 from The Linux Foundation also includes a comprehensive look at other cloud platforms and tools that many sysadmins would be wise to pick up. Mirantis and other vendors, such as Red Hat, also offer certified OpenStack administrator curriculum.

Finally, scripting and development skills can also expand a sysadmin’s horizons and fit in with organizational DevOps goals. Scripting skills, from Python to Perl, are a valuable part of sysadmin’s toolkit. The Linux Foundation offers coursework in this area, too, including Developing Applications for Linux and Linux Performance Tuning. Additionally, The Foundation offers an Introduction to DevOps course that is worth exploring.

In the next article, we will explore specific professional certifications and relevant training to help you move to the next level.

Learn more about essential sysadmin skills: Download the Future Proof Your SysAdmin Career ebook now.

Future Proof Your SysAdmin Career: New Networking Essentials

Future Proof Your SysAdmin Career: Locking Down Security

Future Proof Your SysAdmin Career: Looking to the Cloud

Future Proof Your SysAdmin Career: Configuration and Automation

This Week in Numbers: Comparing Corporate Open Source Contributions on GitHub Organizations

By

The New Stack

-

August 28, 2017

Another way to evaluate GitHub organizations is based on their activity. Open Hub data indicates that 61 percent of the most active organizations on GitHub organizations are commercial enterprises. Most of these companies are working on projects where almost all of the contributors are also employees. Non-profit organizations like those supporting Linux and Kubernetes on average have the highest number of commits. Education organizations have the fewest because many of the projects they maintain are just ways to manage syllabi and homework assignments.

Using Ansible for Deploying Serverless Applications

By

OpenSource.com

-

August 28, 2017

Ansible is designed as the simplest deployment tool that actually works. What that means is that it’s not a full programming language. You write YAML templates that define tasks and list whatever tasks you need to automate your job.

Most people think of Ansible as a souped-up version of “SSH in a ‘for’ loop,” and that’s true for simple use cases. But really Ansible is about tasks, not about SSH. For a lot of use cases, we connect via SSH but also support things like Windows Remote Management (WinRM) for Windows machines, different protocols for network devices, and the HTTPS APIs that are the lingua franca of cloud services.

In a cloud, Ansible can operate on two separate layers: the control plane and the on-instance resources. The control plane consists of everything not running on the OS.

Monitoring Network Traffic More Efficiently

By

MIT News

-

August 28, 2017

In today’s data networks, traffic analysis — determining which links are getting congested and why — is usually done by computers at the network’s edge, which try to infer the state of the network from the times at which different data packets reach their destinations.

If the routers inside the network could instead report on their own circumstances, network analysis would be much more precise and efficient, enabling network operators to more rapidly address problems. To that end, router manufacturers have begun equipping their routers with counters that can report on the number of data packets a router has processed in a given time interval.

But raw number counts are only so useful, and giving routers a special-purpose monitoring circuit for every new measurement an operator might want to make isn’t practical. The alternative is for routers to ship data packets to outside servers for more complex analysis, but that technique doesn’t scale well.