NorNet: An Open Source Inter-Continental Internet Testbed

By

-

February 14, 2017

With new devices and applications creating interesting use cases for IoT, smart cities, vehicle networks, mobile broadband and more, we are creating new ways to use networked technologies, and we need to be able to test these in realistic settings across locations. In his LinuxCon Europe talk, Thomas Dreibholz, Senior Research Engineer at Simula Research Laboratory talked about how they are building NorNet using open source software as an inter-continental Internet testbed for a variety of networked applications.

Dreibholz talked about two concepts underlying the NorNet project, multi-homing and multi-path transport. Multi-homing is designed to make the network more robust by connecting to multiple ISPs at the same time to provide multiple interfaces (addresses) and redundancy for continuous communication, even when some paths fail. Multi-path can make more efficient use of multi-home to utilize paths simultaneously for better throughput using Multi-Path TCP (MPTCP) or Concurrent Multi-Path Transfer for SCTP (CMT-SCTP).

Unfortunately, CMT-SCTP is not available for Linux, so if you want to use CMT-SCTP Dreibholz recommends using FreeBSD; however, MPTCP is available for Linux. While it isn’t yet in the mainline kernel, it has an active community and can be downloaded from the Université catholique de Louvain. The downside is that you need to compile it yourself, and you’ll need to spend a bit of time thinking about how the routing should be configured to take advantage of MPTCP.

The wired part of NorNet Core has 21 sites. Although 11 of the sites are in Norway, the other 10 are geographically distributed, with four in Germany, two in China, and others distributed all over the world. Sites span from Longyearbyen — about 1,200 kilometers away from the North Pole — to a southern site in Sydney, Australia.

Things can go wrong with experimental software and as a result of Murphy’s Law, things will go wrong, Dreibholz says. With sites in hard to reach locations, they are using KVM-based virtual machines running on a very lightweight installation of Ubuntu Server Linux that allow them to replace or reinstall VMs with misbehaving software without needing someone physically present at the site.

For more details about all of the many technologies used in the NorCore testbed or to learn more about getting access to use the testbed for your experiments, watch the entire talk below!

Interested in speaking at Open Source Summit North America on September 11 – 13? Submit your proposal by May 6, 2017. Submit now>>

Not interested in speaking but want to attend? Linux.com readers can register now with the discount code, LINUXRD5, for 5% off the all-access attendee registration price. Register now to save over $300!

Software-Defined Storage Opens Up: 10 Projects to Know

By

Sam Dean

-

February 14, 2017

Throughout 2016, the SDS (Software-Defined Storage) category achieved many new milestones and became increasingly tied to successful cloud deployments. With SDS, organizations can manage policy-based provisioning and management of data storage independent of the underlying hardware. They can also deploy free and open source SDS solutions. Many people are familiar with Ceph and are leveraging it within their OpenStack deployments, but Ceph is far from the only relevant open source SDS project.

A market research report from Gartner predicted that by 2019, 70 percent of existing storage array solutions will be available as a “software only” version. The research firm also predicted that by 2020, 70 percent to 80 percent of unstructured data will be stored in less expensive storage hardware managed by SDS systems.

Just recently, Dell EMC joined the OpenSDS Project, of which The Linux Foundation is the steward. The OpenSDS community seeks to address software-defined storage integration challenges with the goal of driving enterprise adoption of open standards. It comprises storage users and vendors, including Fujitsu, Hitachi Data Systems, Huawei, Oregon State University and Vodafone. The project also seeks to collaborate with other upstream open source communities such as the Cloud Native Computing Foundation, Docker, OpenStack and Open Container Initiative.

According to the Open SDS project’s home, 2017 will be a milestone year for SDS: “The community hopes to have an initial prototype available in Q2 2017 with a beta release by Q3 2017. The initial participants expect OpenSDS will leverage open source technologies, such as Cinder and Manila from the OpenStack community, to best enable support across a wide range of cloud storage solutions.”

Meanwhile, the number of projects in the SDS category is ballooning. They range from Apache Cassandra to Ceph. The Linux Foundation recently released its 2016 report “Guide to the Open Cloud: Current Trends and Open Source Projects,” which provides a comprehensive look at the state of open cloud computing, and includes a section on SDS. You can download the report now, and one of the first things to notice is that it aggregates and analyzes research, illustrating how trends in containers, SDS, and more are reshaping cloud computing. The report provides descriptions and links to categorized projects central to today’s open cloud environment.

In this series of articles, we are calling out many of these projects from the guide, providing extra insights on how the categories are evolving. Below, you’ll find a collection of several important SDS projects and the impact that they are having, along with links to their GitHub repositories, all gathered from the Guide to the Open Cloud:

Software-Defined Storage

Apache Cassandra

Apache Cassandra is a scalable, high-availability database for mission-critical applications. It runs on commodity hardware or cloud infrastructure and replicates across multiple data centers for lower latency and fault tolerance. Cassandra on GitHub

Ceph

Ceph is Red Hat’s distributed, highly scalable block, object, and file storage platform for enterprises deploying public or private clouds. It’s commonly used with OpenStack. Ceph on GitHub

CouchDB

CouchDB, an Apache Software Foundation project, is a single-node or clustered database management system. It provides a RESTful HTTP API for reading and updating database documents. CouchDB on GitHub

Docker Volume Plugins

Docker Engine volume plugins enable Engine deployments to be integrated with external storage systems and enable data volumes to persist beyond the lifetime of a single Engine host. Volume plugins exist for multiple external storage systems including Azure File Storage, NetApp, VMware vSphere, and more. You can find individual plugins on GitHub.

GlusterFS

Gluster is Red Hat’s scalable network filesystem and data management platform. It can deploy on-premise, in private, public, or hybrid clouds, and in Linux containers for media streaming, data analysis, and other data- and bandwidth-intensive tasks. GlusterFS on GitHub

MongoDB

MongoDB is a high performance document database designed for ease of development and scaling. MongoDB on GitHub

Nexenta

NexentaStor is a scalable, unified software-defined file and block storage service that includes data management functionality. It integrates with VMware and supports Docker and OpenStack. Nexenta on GitHub

Redis

Redis is an in-memory data structure store, used as database, cache and message broker. It supports multiple data structures and has built-in replication, Lua scripting, LRU eviction, transactions and different levels of on-disk persistence. Redis on GitHub

Riak CS

Riak CS (Cloud Storage) is object storage software built on top of Riak KV, Basho’s distributed database. It provides distributed cloud storage at any scale, and can be used to build public or private cloud architectures or as storage infrastructure for heavy-duty applications and services. Its API is Amazon S3 compatible and supports per-tenant reporting for use cases involving billing and metering. Riak CS on GitHub

Swift

Swift is OpenStack’s object storage system designed to store and retrieve unstructured data with a simple API. It’s built for scale and optimized for durability, availability, and concurrency across the entire data set. Swift on GitHub

Learn more about trends in open source cloud computing and see the full list of the top open source cloud computing projects. Download The Linux Foundation’s Guide to the Open Cloud report today!

NorNet — Building an Inter-Continental Internet Testbed Based on Open Source Software

By

The Linux Foundation

-

February 14, 2017

Thomas Dreibholz, Senior Research Engineer at Simula Research Laboratory, describes how his team is using open source software to build NorNet — an inter-continental Internet testbed for a variety of networked applications.

Introducing Docker Secrets Management

By

Docker Blog

-

February 14, 2017

A critical element of building safer apps is having a secure way of communicating with other apps and systems, something that often requires credentials, tokens, passwords and other types of confidential information—usually referred to as application secrets. We are excited to introduce Docker Secrets, a container native solution that strengthens the Trusted Delivery component of container security by integrating secret distribution directly into the container platform.

Baidu’s Deep Learning Framework Adopts Kubernetes

By

InfoWorld

-

February 14, 2017

PaddlePaddle, Baidu’s open source framework for deep learning, is now compatible with the Kubernetes cluster management system to allow large models to be trained anywhere Kubernetes can run.

This doesn’t simply expand the range of systems that can be used for PaddlePaddle training; it also provides end-to-end deep learning powered by both projects.

Taking Control of the Programmable Network

By

SDx Central

-

February 14, 2017

Analytics plays a key role in digital-ready networks. It reveals rich contextual insights about users, applications, devices, and threats. This helps organizations and their IT professionals make more informed decisions. To make this happen, however, organizations must do two things. First, they must liberate IT time and resources by automating daily networking tasks, which makes room to focus on business innovation. Hence the willingness to take up SDN and NFV.

Second, organizations must build key programming skills in their network engineers.

How to Speed up Your Apache Web Server and Your Website Page Speed

By

-

February 14, 2017

In today’s fast-paced online world, speedy loading times are almost considered to be a given. Websites that take a long time to load or lag in between pages are often left behind by the vast majority of Internet users, which is why optimizing this aspect of the visitor’s experience is essential for long-term success.

Apache, currently one of the world’s most widely-used web servers, was not expressly designed to set benchmark records, but it can nevertheless handle an impressive number of requests from remote clients and provide a high level of performance if administrators take the time to implement the following five tips and tricks:

1. Always keeping Apache updated to its latest version

Like any piece of software, Apache will work best if upgraded to its latest version. Everything from bug fixes to general improvements will be included in these updates, so it’s worth taking the time to download and install them for your system of choice.

2. Upgrading to a newer version of Kernel

From 2.4 onwards, Kernel supports a sendfile system that allows for high performance network transfers, therefore enabling Apache to deliver static content much faster. For this reason, it pays to upgrade as soon as possible, even if the actual process isn’t exactly friendly for beginners and requires a bit of in-depth knowledge about the internals of Linux.

3. Choosing a Multi-processing Module that works for you

Multi-processing modules allow you to decide how to configure the web server, an important functionality that cannot be neglected at an admin level. Apache currently offers three MPMs to choose from. There’s the “prefork”, the “worker” and the “event”. Study each to get to know their respective advantages and disadvantages, and then choose the one that works best for your particular situation.

4. Allocate the appropriate amount of RAM

Out of all the hardware items that must be taken into account when optimizing your Apache process, RAM is by far the most important. While you cannot control RAM directly, you can limit the number of child processes through the MaxRequestWorkers directive, which will set a limit on RAM usage by Apache. Be sure to always keep RAM usage within limits and never rely on swap because it negatively impacts performance levels.

5. Know your applications

Finally, in order to avoid overburdening your system, be sure to refrain from loading any Apache modules that are not strictly necessary for your application to work. In order to do this, you’ll need to know which applications are running on your server, and disable the modules using the procedures for CentOS and Debian respectively.

As you can see, the aforementioned five tips can make a massive difference when it comes to increasing your Apache web server’s performance. Of course, optimizing performance without also increasing website safety is pointless, so take care to implement adequate security measures as well.

Now, if you are like the vast majority of website owners these days, you probably have your own site on platforms like WordPress, Drupal, Joomla, Magento or SITE123. These were all designed to be as SEO-friendly as possible, but that doesn’t mean they are immune to slow loading times. As always, a badly-run page can be seen as a sign of unprofessionalism, regardless of which website platform you use. So be sure to boost your site’s load speed by employing these essential strategies:

· Use a CDN (Content Delivery Network)

· Use a caching plugin

· Add Expires headers to leverage browser caching

· Clean up your database

· Compress your website with gzip

· Fix all broken links

· Reducing Your redirects

· Minify your CSS and JS files

· Replace PHP with static HTML where possible

· Link to your stylesheets, don’t use @import

· Specify image dimensions

· Put CSS at the top and JS at the bottom

· Disable hotlinking of images

· Switch off all plugins you don’t use

· Minimize round trip times (RTTs)

That concludes our quick rundown of the most important things you can do to boost the loading speed of your website or blog. Nowadays especially, with mobile Internet usage becoming the norm, people are becoming less patient with sites that take forever to load. So make sure that you do everything in your power to keep your website running smoothly and efficiently, and you’ll quickly reap the rewards of your efforts.

What’s The Best Path To Becoming A Data Scientist?

By

Forbes

-

February 14, 2017

There’s a lot of interest in becoming a data scientist, and for good reasons: high impact, high job satisfaction, high salaries, high demand. A quick search yields a plethora of possible resources that could help — MOOCs, blogs, Quora answers to this exact question, books, Master’s programs, bootcamps, self-directed curricula, articles, forums and podcasts. Their quality is highly variable; some are excellent resources and programs, some are click-bait laundry lists. Since this is a relatively new role and there’s no universal agreement on what a data scientist does, it’s difficult for a beginner to know where to start, and it’s easy to get overwhelmed.

VC Investor Martin Casado on the Future of Software-Defined Networking

By

Libby Clark

-

February 13, 2017

Software-defined networking’s biggest accomplishment last year was achieving market traction and validation, says Martin Casado, a general partner at the venture capital firm Andreessen Horowitz. But there are still many challenges ahead for the industry at large and the organizations that aim to drive SDN forward.

Martin Casado is a general partner at Andreessen Horowitz.

“We’ve seen a lot of progress in SDN over the last few years, (but) there is still a lot more work to do,” said Casado, who was previously the co-founder and chief technology officer at Nicira, which was acquired by VMware in 2012. “That said, I’m optimistic that the tangible opportunity will continue to be a strong draw given continued market maturation.”

Casado will elaborate on these ideas and more at Open Networking Summit, April 3-6 in Santa Clara, where he will give a keynote on “The Future of Networking.” Here, he discusses where software-defined networking is headed, the momentum in open source networking projects, challenges they will face, and the best way for companies to get involved in the SDN revolution.

Linux.com: What’s your advice to individuals and companies getting started in SDN?

Martin Casado: Don’t get lost in the noise. While definitions vary, most would agree that SDN involves moving networking functions to a software domain, which changes how it can be created, consumed, and delivered. Yet, like many hyped movements, the term has also been diluted to the point of causing real confusion for those who have not been with the movement long.

I would recommend learning about SDN from a vendor-neutral source, and then determining what value you can get from it, whether as a developer, a vendor, or a user. Then I would align with projects that reinforce your objectives and not spend too much time worrying about every project, product or organization that is being thrown into the SDN bucket.

Linux.com: What have been the biggest successes in SDN in the past year, and what do you expect the industry to accomplish in 2017?

Casado: Market traction and validation were the big takeaways from 2016. The network virtualization space continues to mature with multiple solutions available and individual products breaking the half-billion-dollar mark on software alone. Further, the SD-WAN space continued to gain traction with a number of companies offering innovative solutions. Finally, we’re seeing a new wave of solutions targeting developing markets such as container networking. Both the size of the markets being addressed and the verticalization in multiple spaces are strong signs of the generality and impact SDN is having in the industry. Meanwhile we continue to see great momentum in open source projects and other efforts that drive innovation and adoption.

Linux.com: What will be the biggest challenges for SDN for 2017?

Casado: Maintaining momentum and focus. The industry at-large can be fickle and easily distracted, and while we’ve seen a lot of progress in SDN over the last few years, there is still a lot more work to do. With so many new, exciting technology trends competing for attention, we as a community need to stay focused and continue to drive SDN forward. That said, I’m optimistic that the tangible opportunity will continue to be a strong draw given continued market maturation.

Linux.com: How do we harmonize all the open source networking initiatives across the entire stack and industry?

Casado: To be frank, I don’t think we should. I’m a huge fan of the amount of chaos you find in early markets: It’s all energy and creativity and exploration. A Darwinist system of many ideas, some of enormous value and others that won’t go anywhere. I prefer many conflicting approaches that cover a broad spectrum of the problem domain than trying to foist order and risk constraining innovation too early on. Ultimately, there will be winners and losers and hopefully those that survive and see widespread adoption win because they are the most useful, not because it was pre-ordained by some governing body.

Linux.com: How can companies and individuals best participate in the ‘Open Revolution’ in networking?

Casado: For individuals, I suggest contributing to a project that speaks to you. There is so much great work being done in open networking—from core research, to large open source frameworks, to projects aimed at social good. Contribution can be at any level; it doesn’t have to be code. Documentation, design, outreach, community organization, and evangelization are all very valuable contributions. For companies, the landscape is a bit more complicated. I’d recommend contributing to relevant open source projects that support the movement of functionality to software. This doesn’t have to be an SDN-specific project, but could be an enabler such as Linux, OpenStack, Kubernetes, etc. I strongly believe these contributions are ultimately in the best interest of the company with respect to customer acquisition, maintaining relevance, and recruiting, and they can be done in a way that doesn’t conflict with existing proprietary or closed solutions.

Learn More

Interested in learning more about open source networking? Sign up now for regular updates about Open Networking Summit and other open networking initiatives from The Linux Foundation.

Linux Security Fundamentals: Estimating the Cost of a Cyber Attack

By

Linux Training Staff

-

February 13, 2017

Start exploring Linux Security Fundamentals by downloading the free sample chapter today. DOWNLOAD NOW

Last week, we learned to begin a risk assessment by first evaluating the feasibility of a potential attack and the value of the assets you’re protecting. These are important steps to determining what and how much security will be required for your system.

You must also then weigh these considerations against the potential business impacts of a security compromise with the costs of protecting them.

Costs – How Much?

It is hard to calculate the Return on Investment that managers need in order to make decisions about how to mitigate a risk. How much value does a reputation have?

Estimating the cost of a cyber attack can be difficult, if not impossible. There is little data on how often various industries suffer from different types of intrusions. Until recent laws were passed, companies would often conceal attacks even from law enforcement.

These factors cause difficulties in making rational decisions about how to address the different risks. Security measures may result in the loss of usability, performance, and even functionality. Often, if usability concerns are not addressed in the design of a secure system, users respond by circumventing security mechanisms.

Still, you can get a good idea of the costs associated with a potential loss of business assets, as well as the costs involved in protecting them, to make an informed decision.

Business Impact

The following questions should be evaluated on a regular basis in order to ensure that the security position is optimal for the environment:

• What is the cost of system repair/replacement?

• Will there be lost business due to disruption?

• How much lost productivity will there be for employees?

• Will there be a loss of current customers?

• Will this cause a loss of future customers?

• Are business partners impacted?

• What is your legal liability?

Security Costs

There are many aspects to the costs associated with securing an IT environment. You should consider all of them carefully:

• Software

• Staff

• Training

• Time for implementation

• Impact to customers, users, workers

• Network, Compute, and Storage resources

• Support

• Insurance.

So far in this series, we’ve covered the types of hackers who might try to compromise your Linux system, where attacks might originate, the kinds of attacks to expect, and some of the business tradeoffs to consider around security. The final two parts of this series will cover how to install and use common security tools: tcpdump, wireshark, and nmap.

Stay one step ahead of malicious hackers with The Linux Foundation’s Linux Security Fundamentals course. Download a sample chapter today!

Read the other articles in the series:

Linux Security Threats: The 7 Classes of Attackers

Linux Security Threats: Attack Sources and Types of Attacks

Linux Security Fundamentals Part 3: Risk Assessment / Trade-offs and Business Considerations

Linux Security Fundamentals Part 5: Introduction to tcpdump and wireshark

Linux Security Fundamentals Part 6: Introduction to nmap