Today, open source development is an integral part of the tech industry, as more and more companies are looking for greater collaboration, flexibility and efficiency within their organizations. With what’s trending in tech constantly changing, the open source model provides companies with the ability to accommodate new technologies and at a rapid speed.
Continuing to gain popularity and value, open source development has led to large-scale change and innovation across different areas of the tech industry. Networking, in particular, is one knowledge area that has evolved due to growing mainstream adoption of the open source model.
Hiring managers seek applicants with networking knowledge.
A leading emergent technology, open source networking separates hardware from software, allowing for rapid innovation, scalability, and cost cutting. With open source networking, businesses are more in control of their networks and have the ability to collaborate more freely, bring products and services to market more quickly and adjust to changing market trends.
For open source professionals, especially those in the market for a new job, familiarizing yourself with networking skills is extremely valuable. According to the 2016 Open Source Jobs Report, open source networking is the second most in-demand knowledge area, trailing only slightly behind cloud technologies like OpenStack and Cloud Stack for the number one slot. Twenty-one percent of hiring managers surveyed cited networking as having the biggest impact on open source hiring.
With that in mind, brushing up on some of the most popular networking skills is a good idea. Below are just a few networking skills that are trending in the open source community and are what employers are looking for on Dice:
OpenDaylight:With the rise of cloud computing and the expansion of data centers, software-defined solutions, like this Linux-based platform, are gaining greater traction in the tech industry. OpenDaylight works to help drive network success, reduce business costs and provide employees with greater flexibility.
Network Security: As companies build out their tech capabilities, one thing that they need to ensure is that their data is protected. For that reason, professionals with network security expertise are in high demand, with more than 3,000 job postings available on Dice on any given day, reflecting a 16 percent year-over-year increase.
Strong working knowledge of fundamental networking technologies:You can’t get anywhere without a basic understanding of core networking technologies like TCP/IP, routing and firewalls. These skills are the bread-and-butter of networking.
As the need for open source technologies continues to expand, more employers will look to networking solutions as a means to accommodate this growth. For this reason, open source networking will continue to weigh heavy on hiring managers’ minds when looking for new open source talent. As an open source professional, remaining up-to-date on the latest networking trends and adding new skills to your knowledge base are key in order to build and grow your career.
Network functions virtualization is all the rage because of the money it can save, and because of the network flexibility it helps afford, but the cable industry is enthused about NFV for yet another, less publicized benefit: the potential NFV creates for improving network security.
“We’re looking at SDN and NFV, and how those technologies can potentially mitigate some of the problems, and be used to put in multiple layers of defense,” said CableLabs’ Principal Architect for Security Steve Goeringer. (SeeCableLabs Sets Up NFV Interop Lab.)
Should hackers find a vulnerability they can exploit, and once a network operator detects it, the operator can implement countermeasure quickly and system-wide, if the network is based on SDN and NFV, Goeringer explained.
You know the saying: fast, cheap, or good, pick two. Uber, Twitter, PayPal, and Hubspot show that you can have all three with Apache Mesos.
Apache Mesos is a cluster manager; it sits between the application layer and the operating system, and deploys and manages applications in large-scale clustered environments. But this dry description doesn’t convey its vast scope for creative and ingenious solutions to large-scale problems.
-Uber uses it to run a cluster of geographically-diverse datacenters.
-Twitter built a system to precisely track and cost resource usage.
– PayPal is building a self-sustaining dynamically-sized cluster, in effect using Mesos to run itself.
– HubSpot uses Mesos to dynamically manage 200 Nginx load balancers.
These are all use cases that go far beyond simple clustering. Watch videos, below, of each company’s use case, presented June 1-2 at MesosCon North America 2016. And see all 55+ recorded sessions for more from MesosCon North America.
Uber: Running Cassandra on Apache Mesos Across Multiple Datacenters
Dr. Abhishek Verma, Uber
Dr. Abhishek Verma, first author of the Google Borg Paper, describes using the Apache Cassandra database and Apache Mesos to build a fluid, efficient cluster of geographically diverse datacenters. The goals of this project are five nines reliability, low cost, and reducing hardware requirements. Mesos allows such flexible resource management that you can co-locate services on the same machine.
Mesos does not support multi-datacenter deployments, but Cassandra does. Cassandra has a mechanism called seeds. Seeds are special nodes that tell new nodes how the join the cluster. A seed node “talks” to other nodes in the Cassandra cluster to get the membership and topology information. Dr. Verma’s team created a new custom seed provider, and this has its own URL. New nodes access this URL and gets the information they need to start up and join the cluster.
Dr. Verma explained “We are trying to move away from our in-house custom deployment system to everything running on Mesos. All our services, all our stateful frameworks, everything is going to run on top of Mesos. We can gain these efficiencies because we co-locate services on the same machine. If you have a CPU intensive service, you can co-locate it with a storage engine which is just requiring a lot of SSG space or a lot of memory.
“This can lead to 30 percent fewer machines as we explored in the Google Borg Paper. In this, what we did is we looked at a shared cluster and we looked at the number of machines in this shared cluster. Then it partitioned the workload into production workload and batch workload. We estimated if we just had one cluster, which was serving the production workload, and one which was just serving the batch workload, how many more machines would we need? It turned out that we would need almost 30 percent more machines in the median cell. We can save all of these machines if we co-locate all of these different services on the same machines.” Watch the full talk, below.
https://www.youtube.com/watch?v=U2jFLx8NNro
Twitter: How We Built a Chargeback System for Twitter’s Compute Platform
Micheal Benedict and Jeyappragash Jeyakeerthi, Twitter
Twitter’s compute platform manages thousands of containers across thousands of hosts. But they had no way of knowing if containers were sized properly and using hardware efficiently, couldn’t tell who owned which containers and services, and couldn’t track resource use by container. Micheal Benedict and Jeyappragash Jeyakeerthi (JJ for short) built their Chargeback system on Mesos to answer these questions. Then they make this data available to users, and put a realistic dollar cost on it. This shows users that there is a real cost, and real money that is wasted when they schedule, for example, a 24-core job to do the work of a 4-core job.
https://www.youtube.com/watch?v=QaaEjJVVd44
Benedict describes their goals as “…to capture all the different resources, every infrastructure system provided. One of the first things we want to talk about is we have three big problems over here. The first one is called resource fluidity. What this essentially means is for example, Aurora Mesos has the notion of the three basic resources. You have cores, memory and disk and users get to define that when they’re launching a job. What’s important over here is to note that there’s no notion of time. When you’re actually doing something like Chargeback, you want to charge people for the resources they use over a period of time. This doesn’t capture it. We launch a job, the job gets killed or it continues to run. One of the first things is we wanted to track the fluidity of these resources across time and we had to do that pretty much for every other infrastructure as well. We also wanted to support additional resources as they get added…”
“The next step is to actually put a unit price or a dollar next to these resources. When Aurora is offering cores as a resource to the customer, that I can acquire 1,000 cores, I need to actually put a dollar next to it so people get an understanding of what’s the impact. This dollar needs to actually be as close to the real cost as possible and this is really where we invested a lot of our time in because there’s always this notion of having funny money which really doesn’t work. People are under the assumption, “It’s just funny money. Why do I care?””
Paypal: Efficient Usage of Apache Mesos Using Container Hibernation
Kamalakannan Muralidharan, Paypal
One of the challenges of operating a Mesos cluster is efficiently allocating your resources. One approach is to schedule applications on demand: take down applications when they’re not used, and bring them back up on demand. Set a timeout, and when the application is not used for that period of time send it into hibernation. Then when a use request comes in spin it back up. Integration with a software load balancer provides necessary capabilities such as events based on traffic pattern, and traffic hits.
https://www.youtube.com/watch?v=OJgmfog0e1w
In his MesosCon presentation, Kamalakannan Muralidharan discusses the flexibility and agility that is possible with Mesos: “Even though Mesos is good at handling a cluster, somebody has to provision and make it available for Mesos to use the cluster. We built a system which can automatically create a cluster and monitor the cluster, make sure that this capacity is there for applications to run. If it finds that it says less capacity, it scales up; if it finds there are not many applications running and capacity is underutilized, it will shrink the cluster. We build a complete, closed-loop system to manage Mesos and its Mesos’ resources itself. After once the resource is given to Mesos, Mesos manages it for applications and application deployments.”
HubSpot: Migrating 200+ Load Balancers into Apache Mesos
Stephen Salinas, HubSpot
HubSpot wanted a better way to manage their 200+ Nginx load balancers. They wanted more efficient use of resources, reliability, easier management, easier monitoring so they could know exactly what was happening anywhere at anytime, service discovery, failover, configuration validation, and very fast provisioning. Right, and why not some unicorns as well? But they did it with Mesos and other open source tools, including ZooKeeper and Docker.
https://www.youtube.com/watch?v=JvuJQmIVdGs
Stephen Salinas discusses some of the benefits of their Mesos-based dynamic load-balancer system: “We condensed I think it was about 225 was the highest limit of that down to less than 10 servers, saving us about 24 grand a month in server costs, which was a pretty cool thing. Then on top of that, now that this deploys through our scheduler, it deploys like any other app would, so upgrading Nginx is the same as deploying a new build of an API, so we can leverage all those same deploy processes, putting blockers on deploys, rolling things back if they go bad, and I could do an Nginx upgrade in under a half an hour instead of going into every server, take it out, upgrade it, put it back in, taking hours and hours and hours at a time, so it’s very fast and very easy to operate on this stuff.
“Then kind of the unseen benefit of this is it gave us a bit of a new view on things as to what we could and couldn’t run in Mesos. Obviously there’s challenges to each thing, but if something very, very static like a load balancer could run in Mesos, why not just everything? Why not cash servers? Why not, obviously people are looking at running SQL in Mesos. There’s lots of things out there that can really benefit from everything that Mesos has to offer as long as you do it in the right way.”
The call for speaking proposals is open for MesosCon Europe. The Apache Mesos community wants to hear from you! Share your knowledge, best practices and ideas with the rest of the community. Submit your proposal now. The deadline to submit proposals is June 17, 2016.
Apache, Apache Mesos, and Mesos are either registered trademarks or trademarks of the Apache Software Foundation (ASF) in the United States and/or other countries. MesosCon is run in partnership with the ASF.
In Docker v1.6, Redhat contributed a way to add metadata to your Docker container images using labels. This was a great idea – a standard way to add stuff like notes or licensing conditions to an image. In fact, the only problem with labels is they aren’t used more (<20% of the time by some estimates). Gareth Rushgrove from Puppet has an interesting presentation on the concept. https://speakerdeck.com/garethr/shipping-manifests-bill-of-lading-and-docker-metadata-and-container
We decided we really needed to add labels to our Microscaling images so we did and here are our thought processes, the tools we wrote and our recommendations for how to do the “right thing”.
Huawei announces completion of SDN Agile Controller certification with Red Hat OpenStack Platform 7 at Huawei’s Beijing SDN Open Lab. This marks the first time Huawei’s SDN controller has been certified for interoperability with a mainstream cloud platform. It is an important step for Huawei’s SDN Integration service in building a multi-vendor certification, construct open, cooperative SDN ecosystem.
In future, Huawei plans to continue cooperation with Red Hat to explore customer needs and the direction of development in related fields, while working together to provide customers with the SDN solution, according to Huawei. Huawei sees a trend in using SDN technology to promote agile service innovation, achieving multi-vendor integration has become a significant pain point for operators.
Last year, conventional wisdom had it that containers were much less secure than virtual machines (VMs)! Why? Because it was easier to break out of the container abstraction layer, which is arguably weaker than VM abstractions supported byhardware VT-x optimization. Once broken loose, malicious code could easily attack the host. Even worse, once the host is compromised, the containers lack their own operating system barriers leaving them basically undefended. Since containers have such thin separating walls; it was easy to paint these back door risks with a broad brush.
Here’s a reality check: Front door attacks and unpatched vulnerabilities are much more likely than these backdoor hacks.
OwnCloud provides Self-hosted file syncing and sharing with a nice web interface and syncing apps for all major mobile and desktop platforms with a functionally like Dropbox, mega, etc. ownCloud makes it easy to access and sync files, calendars, and contacts across your devices. In this tutorial, I will guide you to create your own sync and share server platform with ownCloud. We will install ownCloud with Nginx as the web server, MariaDB (a MySQL fork) as the database and php-fpm to handle the request for php on Linux OpenSUSE Leap 42.1.
It’s Sunday, again, so Linus Torvalds has announced the general availability of a new RC (Release Candidate) build of the upcoming Linux 4.7 kernel, which users can take for a test drive as we speak.
Linux 4.7-rc3 is the third Release Candidate version in the development cycle of Linux kernel 4.7, the next major kernel branch, which should see the light of day sometime in mid-July. And, according to Mr. Torvalds, Linux kernel 4.7 RC3 appears to be a fairly quite release whose only major change is a fix for a pending NFS issue.
“Nothing particularly odd has been going on. As promised, rc3 has the fix for the NFS issue that was pending last rc. Not that anybody seems to have noticed (also as expected),” said Linus Torvalds. “The diffstat looks fairly normal and innocuous. But it all is pretty small.”
EMC, a company that’s built much of its business on storage area networks (SAN), has created a connection technology – libStorage – to provide a standardized protocol for containers to interface with stored data. EMC is open sourcing libStorage on GitHub.
“There are a few factors preventing adoption of containers; one of them is storage,” says Josh Bernstein, VP of technology with EMC. “Customers have struggled with how they run persistent apps in containers. It’s a pain point, and now we’re getting to an acknowledgement by industry that this is a problem.”
In this article we shall discuss about two powerful Linux utilities that are used for sharing file/folder to network computers that are scp and rsync. scp allows you to simply copy directories or files from/to any remote destination whereas the rsync, besides simply copying files also acts as a synchronising tool that would synchronise the changes between source and destination. The discussion below is about their usage and various parameters that can be used along with to increase speed and security.
In this article we shall discuss about two powerful Linux utilities that are used for sharing file/folder to network computers that are scp and rsync. scp allows you to simply copy directories or files from/to any remote destination whereas the rsync, besides simply copying files also acts as a synchronising tool that would synchronise the changes between source and destination. The discussion below is about their usage and various parameters that can be used along with to increase speed and security.