First, if you are not sure what Kafka is, see this article.
Kafka consists of records, topics, consumers, producers, brokers, logs, partitions, and clusters. Records can have keys (optional), values, and timestamps. Kafka records are immutable. A Kafka Topic is a stream of records ("/orders", "/user-signups"). You can think of a topic as a feed name. A topic has a log which is the topic’s storage on disk. A topic log is broken up into partitions and segments. The Kafka Producer API is used to produce streams of data records. The Kafka Consumer API is used to consume a stream of records from Kafka. A broker is a Kafka server that runs in a Kafka cluster. Kafka brokers form a cluster. The Kafka cluster consists of many Kafka brokers on many servers. Broker some times refer to more of a logical system or as Kafka as a whole.
Kafka uses ZooKeeper to manage the cluster. ZooKeeper is used to coordinate the brokers/cluster topology. ZooKeeper is a consistent file system for configuration information. ZooKeeper is used for leadership election for broker topic partition leaders.
Unlike servers or networking equipment, which are typically hacked through isolated access points and exist in sandboxed and supervised environments, IoT devices are more vulnerable to malevolent threat actors.
The underlying risk to a larger network
If one device is compromised, it’s next to impossible for a vendor to issue an OTA and update millions of devices. An insecure device in a network is enough to put the whole network and the devices connected to it in jeopardy: servers, smartphones, and desktops in addition to IoT devices, letting a single device to compromise confidential data from bank and health information.
Vendors are startups and SMBs
IoT companies are mostly startups and SMBs and have enough fund only to back their R&D and marketing team. They don’t have the funds to hire an army of security experts and ethical hackers to ensure secure deployments. As an alternative, they bake security features in IoT’s software environment that is often outsourced to a enterprises app development company in Asia Pacific to cut cost and boost margin in a hyper competitive market. Many of the IoT devices on today’s shelves are by necessity inexpensive to manufacture, which means companies are less likely to spend high dollar on security throughout the development process.
The security Surprises
I do laundry every Sunday afternoon. One fine day my smart washing machine did not work. I unlocked my iPhone and tried to reset it; it did not. The very moment the machine sent me a notification “We hacked your machine and it will only work with our detergent now”. So I ordered a year’s supply of the detergent even when I did not want to.
This did not happen with me. But it might.
A few months ago, a group of naysayers hijacked a smart refrigerator to make it show pornographic spam while making ice cubes. Baby monitors can be turned into eavesdropping devices. CIA tools can hack IoT devices, such as Samsung SmartTVs, and turn them into a bugging device.
Gartner, Inc. forecasts that 6.4 billion connected things are in use worldwide in 2016 and will reach 20.8 billion by 2020.
The two categories of security threats
Threats to IoT can be divided into two categories. First, devices are taken over to do something they are not intended to do, like a home surveillance system that uploads photos to Instagram every minute
Second, devices can be commandeered to do precisely what they are supposed to do but with a catch. A hijacked Nest will set room temperature beyond human tolerance limit.
Vendors are doing their bit from putting end point security suites to data encryption. The question is, is this enough?
1. The everyday security
Defending and fortifying the network connecting IoT devices to back-end systems over the internet, IoT network security, owing to lack of common standards and protocols unlike traditional network security devices poses significant problems and larger difficulties.
An everyday endpoint security suite with antivirus, antispyware and firewalls and intrusion prevention and detection systems shall help.
2. Passwords, biometrics and beyond
Letting users to authenticate IoT devices from a username and password combination to more sophisticated verification mechanisms such as two-way authentication, digital certificates, retina and fingerprint scanners.
The catch is IoT authentication scenarios are not man-to-machine but machine-to-machine. Biometric methods can only stop a person from accessing the IoT device, not another machine or ‘talking’ smart device.
3. Encrypting data and communication
Encrypting data stored in an IoT device, on cloud and in transit will ensure that an unsolicited device or human can’t get access to it. Standard cryptographic algorithms ensure data integrity and avert data sniffing by hackers.
Again, IoT fragmentation confines standard encryption methods and procedures in IoT implementation.
4. Public Key Infrastructure
Offering comprehensive cryptographic key and life-cycle capabilities, X.509 digital certificate and providing public/private key generation, delivery, supervision, and cancelation.
The specification of low-cost IoT devices may confine or avert possibility to employ Public Key Infrastructure.
Vendors for the sake of security must start including Digital certificates hardcoded to IoT devices at the time of distribution and letting an authorized third party IoT device activate it at the time of authentication. DigiCert, and Gemalto can help.
5. AI and Machine Learning
Gathering, combining, monitoring, and standardizing data from various IoT devices in a system and offering disallowed reporting and warning on specific events, particularly when they fall outside the purview of established policies.
Artificial intelligence assisted data mining will give way to more analytical modeling and glitch detection, but these abilities are still evolving.
IoT security analytics will gradually be needed to sense IoT-specific attacks and intrusions that are not recognized by old-school intrusion prevention system like an end point security suite.
6. API Security
The data to and from IoT devices must be authenticated before acquiring any actionable value. The data can be from a database, and mobile apps using familiar REST-based APIs.
API security will ensure only approved devices, 3rd party vendors, and mobile application are interacting with APIs, perceiving upcoming threats and attacks against particular APIs, defending the integrity of data in transit.
Will my washing machine ask for a ransom?
The ongoing evolution of IoT-specific security threats will certainly force innovation in this space. Newer IoT-specific security technologies will appear soon.
It’s authoritative for digital businesses today to balance the business goals that IoT-connected products can bring with the recognition that the smart devices have turn out to be a striking attack plane for pranksters, hackers and even cybercriminals looking to create nuisance, disruption and gain access to sensitive data.
Remember when you first started out with Linux? Depending on the environment you’re coming from, the learning curve can be somewhat challenging. Take, for instance, the number of commands found in /usr/bin alone. On my current Elementary OS system, that number is 1,944. Of course, not all of those are actual commands (or commands I would use), but the number is significant.
Because of that (and many other differences from other platforms), new users (and some already skilled users) need a bit of help now and then.
For every administrator, there are certain skills that are must-have:
Understanding of the platform
Understanding commands
Shell scripting
When you seek out assistance, sometimes you’ll be met with RTFM (Read the Fine/Freaking/Funky Manual). That doesn’t always help when you have no idea what you’re looking for. That’s when you’ll be glad for apps like Guide to Linux.
Unlike most of the content you’ll find here on Linux.com, this particular article is about an Android app. Why? Because this particular app happens to be geared toward helping users learn Linux.
And it does a fine job.
I’m going to give you fair warning about this app—it’s not perfect. Guide to Linux is filled with broken English, bad punctuation, and (if you’re a purist) it never mentions GNU. On top of that, one particular feature (one that would normally be very helpful to users) doesn’t function enough to be useful. Outside of that, Guide to Linux might well be one of your best bets for having a mobile “pocket guide” to the Linux platform.
With this app, you’ll enjoy:
Offline usage.
Linux Tutorial.
Details of all basic and advanced Linux commands of Linux.
Includes command examples and syntax.
Dedicated Shell Script module.
On top of that, Guide to Linux is free (although it does contain ads). If you want to get rid of the ads, there’s an in-app purchase ($2.99 USD/year) to take care of that.
Let’s install this app and then take a look at the constituent parts.
Installation
Like all Android apps, installation of Guide to Linux is incredibly simple. All you have to do is follow these easy steps:
Open up the Google Play Store on your Android device
Search for Guide to Linux
Locate and tap the entry by Essence Infotech
Tap Install
Allow the installation to complete
Figure 1: The Guide to Linux main window.
Once installed, you’ll find the launcher for Guide to Linux in either your App Drawer or on your home screen (or both). Tap the icon to launch the app.
Usage
Let’s take a look at the individual features that make up Guide to Linux. You will probably find some features more helpful than others, and your experience will vary. Before we break it down, I’ll make mention of the interface. The developer has done a great job of creating an easy-to-use interface for the app.
From the main window (Figure 1), you can gain easy access to the four features.
Tap any one of the four icons to launch a feature and you’re ready to learn.
Figure 2: The tutorial begins at the beginning.
Tutorial
Let’s start out with the most newbie-friendly feature of the app—Tutorial. Open up that feature and you’ll be greeted by the first section of the tutorial, “Introduction to the Linux Operating System” (Figure 2).
If you tap the “hamburger menu” (three horizontal lines in the top left corner), the Table of Contents are revealed (Figure 3), so you can select any of the available sections within the Tutorial.
Figure 3: The Tutorial Table of Contents.
Unless you haven’t figured it out by now, the Tutorial section of Guide to Linux is a collection of short essays on each topic. The essays include pictures and (in some cases) links that will send you to specific web sites (as needed to suit a topic). There is no interaction here, just reading. However, this is a great place to start, as the developer has done a solid job of describing the various sections (grammar notwithstanding).
Although you will see a search option at the top of the window, I haven’t found that feature to be even remotely effective—but it’s there for you to try.
For new Linux users, looking to add Linux administration to their toolkit, you’ll want to read through this entire Tutorial. Once you’ve done that, move on to the next section.
Commands
The Commands feature is like having the man pages, in hand, for many of the most frequently used Linux commands. When you first open this, you will be greeted by an introduction that explains the advantage of using commands.
Figure 4: The Commands sidebar allows you to check out any of the listed commands.
Once you’ve read through that you can either tap the right-facing arrow (at the bottom of the screen) or tap the “hamburger menu” and then select the command you want to learn about from the sidebar (Figure 4).
Tap on one of the commands and you can then read through the explanation of the command in question. Each page explains the command and its options as well as offers up examples of how to use the command.
Shell Script
At this point, you’re starting to understand Linux and you have a solid grasp on commands. Now it’s time to start understanding shell scripts. This section is set up in the same fashion as is the Tutorial and Commands sections.
You can open up a sidebar Table of Contents to take and then open up any of the sections that comprise the Shell Script tutorial (Figure 5).
Figure 5: The Shell Script section should look familiar by now.
Once again, the developer has done a great job of explaining how to get the most out of shell scripting. For anyone interested in learning the ins and outs of shell scripting, this is a pretty good place to start.
Terminal
Now we get to the section where your mileage may vary. The developer has included a terminal emulator with the app. Unfortunately, when installing this on an unrooted Android device, you’ll find yourself locked into a read-only file system, where most of the commands simply won’t work. However, I did install Guide to Linux on a Pixel 2 (via the Android app store) and was able to get a bit more usage from the feature (if only slightly). On a OnePlus 3 (not rooted), no matter what directory I change into, I get the same “permission denied” error, even for a simple ls command.
On the Chromebook, however, all is well (Figure 6). Sort of. We’re still working with a read-only file system (so you cannot actually work with or create new files).
Figure 6: Denied!
Remember, this isn’t actually a full-blown terminal, but a way for new users to understand how the terminal works. Unfortunately, most users are going to find themselves frustrated with this feature of the tool, simply because they cannot put to use what they’ve learned within the other sections. It might behoove the developer to re-tool the terminal feature as a sandboxed Linux file system, so users could actually learn with it. Every time a user would open that tool, it could revert to its original state. Just a thought.
In the end…
Even with the terminal feature being a bit hamstrung by the read-only filesystem (almost to the point of being useless), Guide to Linux is a great tool for users new to Linux. With this guide to Linux, you’ll learn enough about Linux, commands, and shell scripting to feel like you have a head start, even before you install that first distribution.
Learn more about Linux through the free “Introduction to Linux” course from The Linux Foundation and edX.
The Linux Foundation has released their anticipated annual report for job seekers and hiring managers containing key info on in-demand skills and outlook. More top stories and open source opinion articles in this week’s digest
1) The Linux Foundation’s 2017 Jobs Report (#OSSJobs) has been released and the findings are illuminating for seekers and hiring managers alike.
3) “Microsoft and Red Hat are expanding the parameters of a relationship first struck in 2015, bringing Azure and SQL Server closer to Red Hat’s container orchestration product.”
I first installed Linux in 1993. I ran MS-DOS at the time, but I really liked the Unix systems in our campus computer lab, where I spent much of my time as an undergraduate university student. When I heard about Linux, a free version of Unix that I could run on my 386 computer at home, I immediately wanted to try it out. My first Linux distribution was Softlanding Linux System (SLS) 1.03, with Linux kernel 0.99 alpha patch level 11. That required a whopping 2MB of RAM, or 4MB if you wanted to compile programs, and 8MB to run X windows.
On August 25, the Linux kernel will reach its 26th anniversary. To celebrate, I reinstalled SLS 1.05 to remind myself what the Linux 1.0 kernel was like and to recognize how far Linux has come since the 1990s. Join me on this journey into Linux nostalgia!
Docker will be at Open Source Summit from to highlight new development with the Moby Project and it’s various components: containerd, LinuxKit, InfraKit, Notary, etc.
Come see us at Booth #510 to learn more about:
The different uses cases for the Moby Projects and components
The difference between Docker and the Moby Project
How to get started with each component
As part of the OSS NA, Docker is also organizing a Moby Summit on September 14, 2017. Following the success of the previous editions, we’ll keep the same format which consists of short technical talks / demos in the morning and Birds-of-a-Feather in the afternoon.
We have an excellent line up of speakers in store for you and are excited to share the agenda below. We hope that these sessions inspire you to come participate in the Moby community and register for this Moby summit.
For those of you who can’t attend the summit we recommend the following sessions as part of the main event / tracks.
Some very useful commands for making life on the command line more rewarding.
Working on the Linux command can be a lot of fun, but it can be even more fun when you use commands that take less work on your part or display information in interesting and useful ways. In today’s post, we’re going to look at half a dozen commands that might make your time on the command line more profitable.
watch
The watch command will repeatedly run whatever command you give it and show you the output. By default, it runs the command every two seconds. Each successive running of the command overwrites what it displayed on the previous run, so you’re always looking at the latest data.
The Linux Foundation is host to more than 100 open source projects, but only a handful are foundations unto themselves. Cloud Foundry Foundation is unique in its standing as a Linux Foundation project: a nonprofit foundation and an open source project that came to the table fully formed. Incepted at VMware in 2010, Cloud Foundry was transferred to Pivotal in 2013 before being open sourced, at which point the Cloud Foundry Foundation was established.
The importance of the Foundation is multifaceted, but its primary significance is it holds all of the intellectual property for Cloud Foundry — and because it is a 501(c)(6), that means the intellectual property can never be transferred back to a for-profit company.
Over the past five years, some 40,000 girls have learned to code through the Girls Who Code’s summer camps and afterschool programs. But Ms. Saujani wanted to expand the group’s reach, and was looking for new ways to recruit girls into the tech industry.
For a tech evangelist, her solution was surprisingly retro and analog: books. Girls Who Code is creating a publishing franchise, and plans to release 13 books over the next two years through a multibook deal with Penguin. The titles range from board books and picture books for babies and elementary school children, to nonfiction coding manuals, activity books and journals, and a series of novels featuring girl coders.
This week, the organization is releasing its first two books — an illustrated nonfiction coding manual by Ms. Saujani, and a novel, “The Friendship Code,” which features a group of girls who become friends in an after-school coding club.
If telecom operators or enterprises were to build their networks from scratch today, they would likely build them as software-defined resources, similar to Google or Facebook’s infrastructure. That’s the premise of Network Functions Virtualization (NFV).
NFV is a once in a generation disruption that will completely transform how networks are built and operated. And, OPNFV is a leading open source NFV project that aims to accelerate the adoption of this technology.
Are you a telecom operator or connected enterprise employee wondering which open source projects might help you with your NFV transformation initiatives? Or a technology vendor attempting to position your products and services in the new NFV world? Or perhaps an engineer, network operator or business leader wanting to progress your career using open source projects (case in point, in 2013 Rackspace stated that network engineers with OpenStack skills made, on average, 13 percent more salary than their counterparts)? If any of this applies to you, the Understanding OPNFVbook is a perfect resource for you.
In 11 easy-to-read chapters and over 144 pages, this book (written by Nick Chase from Mirantis and me) covers an entire range of topics from an overview of NFV, NFV transformation, all aspects of the OPNFV project, to VNF onboarding. After reading this book, you will have an excellent high-level understanding of what OPNFV is and how it can help you or your organization. This book is not specifically meant for developers, though it may be useful for background information. If you are a developer looking to get involved in a specific OPNFV project as a contributor, then wiki.opnfv.org is still the best resource for you.
In this blog series, we will give you a flavor of portions of the book — in terms of what’s there and what you might learn.
Let’s start with the first chapter. Chapter 1, no surprise, provides an introduction to NFV. It gives a super-brief overview of NFV in terms of business drivers (the need for differentiated services, cost pressures and need for agility), what NFV is and what benefits you can expect from NFV.
Briefly, NFV enables complex network functions to be performed on compute nodes in data centers. A network function performed on a compute node is called a Virtualized Network Function (VNF). So that VNFs can behave as a network, NFV also adds the mechanisms to determine how they can be chained together to provide control over traffic within a network.
Although most people think of it in terms of telecommunications, NFV encompasses a broad set of use cases, from Role Based Access Control (RBAC) based on application or traffic type, to Content Delivery Networks (CDN) that manage content at the edges of the network (where it is often needed), to the more obvious telecom-related use cases such as Evolved Packet Core (EPC) and IP Multimedia System (IMS).
Additionally, some of the main benefits include increased revenue, improved customer experience, reduced operational expenditure (OPEX), reduced capital expenditures (CAPEX) and freed-up resources for new projects. This section also provides results of a concrete NFV total-cost-of-ownership (TCO) analysis. Treatment of these topics is brief since we assume you will have some NFV background; however, if you are new to NFV, not to worry — the introductory material is adequate to understand the rest of the book.
The chapter concludes with a summary of NFV requirements — security, performance, interoperability, ease-of-operations and some specific requirements such as service assurance and service function chaining. No NFV architecture or technology can be truly successful without meeting these requirements.
After reading this chapter, you will have a good overview of why NFV is important, what NFV is, and what is technically required to make NFV successful. We will look at following chapters in upcoming blog posts.
This book has proven to be our most popular giveaway at industry events and a Chinese version is now under development! But you can download the eBook in PDF right now, or order a printed version on Amazon.
