5 Videos to Get You Pumped to Speak at MesosCon 2017

By

-

February 22, 2017

Last year, experts from Uber, Twitter, PayPal, and Hubspot, and many other companies shared how they use Apache Mesos at MesosCon events in North America and Europe. Their talks helped inspire developers to get involved in the project, try out an installation, stay informed on project updates, and generally get pumped to use and participate in Apache Mesos.

The MesosCon program committee is now seeking proposals from speakers with fresh ideas, enlightening case studies, best practices, or deep technical knowledge to share with the Apache Mesos community again this year. MesosCon is an annual conference held in three locations around the globe and organized by the Apache Mesos community in partnership with The Linux Foundation.

March 25 is the deadline for speakers to submit proposals for MesosCon Asia. MesosCon North America’s deadline is May 20 and MesosCon Europe’s is July 8. Here, we’ve rounded up the top 5 videos from the 2016 MesosCon North America event for some inspiration. Submit your speaking proposal now!

1. How Verizon Labs Built a 600 Node Bare Metal Mesos Cluster in Two Weeks

Craig Neth, Distinguished Member of the Technical Staff at Verizon Labs, describes building a 600-node Mesos cluster from bare metal in two weeks. His team didn’t really get it all done in two weeks, but it’s a fascinating peek at some ingenious methods for accelerating the installation and provisioning of the bare hardware, and some advanced ideas on hardware and rack architectures.

https://www.youtube.com/watch?v=6P8htQnXCfM?list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC

2. 4 Unique Ways Uber, Twitter, PayPal, and Hubspot Use Apache Mesos

Dr. Abhishek Verma, first author of the Google Borg Paper, describes how Uber used the Apache Cassandra database and Apache Mesos to build a fluid, efficient cluster of geographically diverse datacenters. The goals of this project were five nines reliability, low cost, and reducing hardware requirements. Mesos allows such flexible resource management that you can co-locate services on the same machine.

https://www.youtube.com/watch?v=U2jFLx8NNro

3. Apache Mesos for Beginners: 3 Videos to Help You Get Started

“How do I get my hands on this? I don’t have a datacenter or a team of engineers. What if I want to become a contributor? How do I make this all go in my own little test lab?”

The talks highlighted in this article will help you answer these questions. Aaron Williams, Joris Van Remoorter, and Michael Park of Mesosphere, and Frank Scholten of Container Solutions share how to run Mesos on a laptop, how to become a contributor, and the basic architecture of a Mesos-based datacenter.

https://www.youtube.com/watch?v=J14_H4T0JB0?list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC

4. Apache Spark Creator Matei Zaharia Describes Structured Streaming in Spark 2.0

Apache Spark has been an integral part of Mesos from its inception. Spark is one of the most widely used big data processing systems for clusters. Matei Zaharia, the CTO of Databricks and creator of Spark, talked about Spark’s advanced data analysis power and new features in its upcoming 2.0 release in his MesosCon 2016 keynote.

https://www.youtube.com/watch?v=L029ZNBG7bk?list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC

5. Open Source Is Key to the Modern Data Center, Says EMC’s Joshua Bernstein

DevOps is key to agility, agility is key to innovation and success, and open source powers DevOps. Joshua Bernstein, Vice President of Technology at EMC, describes the value that this brings to an organization: “We automate everything. We drive out corner cases. We strive for commodity hardware…The biggest thing is that we value this ability to interoperate. This goes along with microservices and the way that we build microservice applications now. We also value tremendously the ability to leverage a collaborative community.”

https://www.youtube.com/watch?v=-Xts6Mkujkg

Submit a proposal to speak at MesosCon Asia » The deadline is March 25.

Submit a proposal to speak at MesosCon North America » The deadline is May 20.

Submit a proposal to speak at MesosCon Europe » The deadline is July 8.

Not interested in speaking but want to attend? Linux.com readers receive 5% off the “attendee” registration with code LINUXRD5.

Register for MesosCon Asia » Save $125 through April 30.

Register for MesosCon North America » Save $200 through July 2.

Register for MesosCon Europe » Save $200 through August 27.

Apache, Apache Mesos, and Mesos are either registered trademarks or trademarks of the Apache Software Foundation (ASF) in the United States and/or other countries. MesosCon is run in partnership with the ASF.

Distributed Logging for Containers

By

Ian Murphy

-

February 22, 2017

The era of microservices calls for a new approach to logging with built-in infrastructure for both aggregation and storage. Multiple applications running in isolated containers require a specialized approach to make sure all data is collected, stored and usable later. Eduardo Silva, a software engineer at Treasure Data, gave a crash course in distributed logging during his keynote at CloudNativeCon last November, showing the pros and cons of different infrastructure models and highlighting the open source project Fluentd.

“When we are talking about this kind of architecture, we need to stop thinking about just a file, just a file system,” Silva said. “It’s about how I’m going to deal with the different aggregation patterns, how I’m going to distribute my logs.”

Silva said there are three main parts of a distributed logging infrastructure: collector nodes, aggregator nodes, and then a destination — a database, a file system, or another service, etc. Collectors retrieve the raw logs from the application and parse their content. Aggregators pull in that log data from multiple sources and then convert the logs — which could be in a number of different formats — into streams. Destinations access the data streams and store the information somewhere permanent.

Depending on variables like CPU resources, network traffic, and whether or not the system needs high availability and/or redundancy, there are different ways to configure a distributed logging system, Silva said. The main question is where to put the aggregator — either in the collection container nodes, if high network traffic is an issue, or closer to the destination, if network failure is likely or data loss is unforgivable.

“To [best] implement all these integration patterns,” Silva said, “you need the right tool for this kind of solution. So this is where Fluentd joins in. Fluentd is an open source data and load collector, which was designed to achieve all these kind of aggregation patterns and adapt to your own needs. It was made with high performance in mind. It has built-in reliability, structured logs, and a pluggable architecture.”

Silva said the native Docker logging driver uses Fluentd, and both Kubernetes and OpenShift use Fluentd as the main logging aggregator. It’s infrastructure includes built-in parsers and filters to handle and convert multiple data types, and buffers to store more intense logging streams in memory to protect against database or network failures. It’s been an active project since 2011.

Silva announced on stage that Fluentd has joined the Cloud Native Computing Foundation as a partner, so the open source project is poised to become an even bigger part of the open source foundation’s work.

“We have thousands of companies using Fluentd,” Silva said. “We have thousands of individual users, and as you saw we have more than 600 plugins around and most of them are made by individuals.”

Watch the complete presentation below:

Want to learn more about Kubernetes? Get unlimited access to the new Kubernetes Fundamentals training course for one year for $199. Sign up now!

Logging for Containers by Eduardo Silva, Treasure Data

By

The Linux Foundation

-

February 22, 2017

Eduardo Silva, a software engineer at Treasure Data, gave a crash course in distributed logging during his keynote at CloudNativeCon last November.

Basic Rules to Streamline Open Source Compliance For Software Development

By

Ibrahim Haddad

-

February 22, 2017

The following is adapted from The Linux Foundation’s e-book, Open Source Compliance in the Enterprise, by Ibrahim Haddad, PhD.

Companies will almost certainly face challenges establishing their open source compliance program. In this series of articles, based on The Linux Foundation’s e-book, Open Source Compliance in the Enterprise, we discuss some of the most common challenges, and offer recommendations on how to overcome them.

The first challenge is to balance the compliance program and its supporting infrastructure with (existing) internal processes while meeting deadlines to ship products and launch services. Various approaches can help ease or solve such challenges and assist in the creation of a streamlined program that is not seen as a burden to development activities.

Companies should streamline open source management upon two important foundational elements: a simple and clear compliance policy and a lightweight compliance process.

Mandate basic rules

It’s important to first have executive-level commitment to the open source management program to ensure success and continuity. In addition, policies and processes have to be light and efficient so that development teams do not regard them as overly burdensome to the development process. Establish some simple rules that everyone must follow:

Require developers to fill out a request form for any open source software they plan to incorporate into a product of software stack.
Require third-party software suppliers to disclose information about open source software included in their deliverables. Your software suppliers may not have great open source compliance practices, and it is recommended that you update your contractual agreement to include language related to open source disclosures.
Mandate architecture reviews and code inspections for the Open Source Review Board (OSRB) to understand how software components are interrelated and to discover license obligations that can propagate from open source to proprietary software. You will need proper tooling to accommodate a large-scale operation.
Scan all incoming software received from third party software providers and ensure that their open source disclosures are correct and complete.

Integrate Rules Into the Existing Development Process

Once the basic rules have been established, the most successful way to create compliance is to incorporate the compliance process and policies, checkpoints and activities as part of existing software development processes.

The priority for all organizations is to ship products and services on time while building and expanding their internal open source compliance infrastructure. Therefore, you should expect to build your compliance infrastructure as you go, keeping in mind scalability for future activities and products. The key is thoughtful and realistic planning.

Plan a complete compliance infrastructure to meet your long- term goals, and then implement the pieces stepwise, as needed for short-term execution.

For instance, if you are just starting to develop a product or deliver a service that includes open source and you do not yet have any compliance infrastructure in place, the most immediate concern should be establishing a compliance team, processes and policy, tools and automation, and training your employees. Having kicked off these activities (in that order) and possessing a good grip on the build system (from a compliance perspective), you can move on to other program elements.

The next challenge to establishing an open source compliance program is clearly communicating your organization’s efforts to meet its open source license obligations with others inside and outside the company. In the next article, we’ll cover some practical ways to approach communication.

Get the open source compliance training you need. Take the free “Compliance Basics for Developers” course from The Linux Foundation. Sign up now!

Read the other articles in this series:

The 7 Elements of an Open Source Management Program: Strategy and Process

The 7 Elements of an Open Source Management Program: Teams and Tools

How and Why to do Open Source Compliance Training at Your Company

Evolution of Business Logic from Monoliths through Microservices, to Functions

By

Adrian Cockcroft

-

February 22, 2017

The whole point of running application software is to deliver business value of some sort. That business value is delivered by creating business logic and operating it so it can provide a service to some users. The time between creating business logic and providing service to users with that logic is the time to value. The cost of providing that value is the cost of creation plus the cost of delivery. …

As technology has progressed over the last decade, we’ve seen an evolution from monolithic applications to microservices and are now seeing the rise of serverless event driven functions, led by AWS Lambda. What factors have driven this evolution? Low latency messaging enabled the move from monoliths to microservices, low latency provisioning enabled the move to Lambda.

Blockchain: A New Hope, or Just Hype?

By

World Economic Forum

-

February 22, 2017

Cryptocurrencies such as bitcoin may have captured the public’s fancy – and also engendered a healthy dose of skepticism — but it is their underlying technology that is proving to be of practical benefit to organizations: the blockchain. Many industries are exploring its benefits and testing its limitations, with financial services leading the way as firms eye potential windfalls in the blockchain’s ability to improve efficiency in such things as the trading and settlement of securities. The real estate industry also sees potential in the blockchain to make homes — even portions of homes — and other illiquid assets trade and transfer more easily. The blockchain is seen as disrupting global supply chains as well, by boosting transaction speed across borders and improving transparency.

5 Open Source Security Tools Too Good to Ignore

By

InfoWorld

-

February 22, 2017

If you haven’t been looking to open source to help address your security needs, it’s a shame—you’re missing out on a growing number of freely available tools for protecting your networks, hosts, and data. The best part is, many of these tools come from active projects backed by well-known sources you can trust, such as leading security companies and major cloud operators. And many have been tested in the biggest and most challenging environments you can imagine.

Open source has always been a rich source of tools for security professionals—Metasploit, the open source penetration testing framework, is perhaps the best-known—but information security is not restricted to the realm of researchers, investigators, and analysts, and neither are the five open source security tools we survey below.

Why Upstream Contributions Matter when Developing Open Source NFV Solutions

By

Vertical Industries Blog

-

February 22, 2017

When software is developed using open source methods, an upstream repository of the code is accessible to all members of the project. Members contribute to the code, test it, write documentation and can create a solution from that code to use or distribute under license. If an organization follows the main stream or branch of the upstream code their solution will receive all the changes and updates created in the upstream repository. Those changes simply “flow down” to the member’s solution. However, if a member organization forks the code — if they create a solution that strays from the main stream — their solution no longer receives updates, fixes and changes from the upstream repository. This organization is now solely responsible for maintaining their solution without the benefit of the upstream community, much like the baby salmon that took a tributary and then have to fend for themselves rather than remain in the main stream and receive the benefit and guidance of the other salmon making their way to the ocean.

Network functions virtualization (NFV) is the revolution sweeping telcos as they modernize their infrastructure. After years of lock-in to proprietary vendors who were solely responsible for the solution…

Apache Kafka: A Primer

By

The New Stack

-

February 22, 2017

Apache Kafka is fast becoming the preferred messaging infrastructure for dealing with contemporary, data-centric workloads such as Internet of Things, gaming, and online advertising. The ability to ingest data at a lightening speed makes it an ideal choice for building complex data processing pipelines. In a previous article, we discussed how Kafka acts as the gateway for IoT sensor data for processing hot path and cold path analytics.

In this article, we will introduce the core concepts and terminology of Apache Kafka along with the high-level architecture.

Top Lightweight Linux Distributions To Try In 2017

By

Mohd Sohail

-

February 22, 2017

Hey friends! Today I am going to discuss the top lightweight Linux distros you can try this year on your computer. Although you got yourself a pretty Linux already but there is always something new to try in Linux. Remember I recommend to try this distro in VirtualBox firstly or with the live boot before messing with your system. All distro that I will mention here will be new and somewhat differ from regular distros.

Read At LinuxAndUbuntu