Home Blog Page 568

Introduction to Functional Programming

Depending on whom you ask, functional programming (FP) is either an enlightened approach to programming that should be spread far and wide, or an overly academic approach to programming with few real-world benefits. In this article, I will explain what functional programming is, explore its benefits, and recommend resources for learning functional programming.

Syntax primer

Code examples in this article are in the Haskell programming language. All that you need to understand for this article is the basic function syntax:

Read more at OpenSource.com

Trivial Transfers with TFTP, Part 2: Configuration

In the first article of this series, we looked at some of the, admittedly limited, features of TFTP along with some reasons why you might want to use it. Here, we take a look at the main config file and how to enable and disable services for improved security.

Debian

The following config for our Debian package won’t apply directly to Red Hat, but much of it should make sense at least. It’s important to pay attention to the openbsd-inetd package, which is provided because that’s how systemd interacts with starting and stopping the tftpd daemon via the old-school inetd service.

If you query Debian Jessie’s installation, then the README file offers the following version information: “This is netkit-tftp-0.17 for Linux.”

The server manual (viewed with the command man in.tftpd) also dutifully informs us that tftpd supports the DARPA Trivial File Transfer Protocol and respects which port it operates on by looking up the /etc/services file. In other words, to move it off of port 69, you can simply edit entries found within that file.

The Debian way of getting your TFTP server to start up after a reboot would look like this (assuming you installed the tftpd package of course and not the one offered as an alternative on Red Hat systems):

# systemctl enable openbsd-inetd

Our main config file (I can hear some of those cobwebs being blown away as you read the location of this file) can be found at: /etc/inetd.conf. This controls all things inetd related, but the helpful tftpd has included a line for us luckily.

Indeed an abbreviated version of that file, for the sake of simplicity, might look like that found in Listing 1:

# Packages should modify this file by using update-inetd(8)
#
# <service_name> <sock_type> <proto> <flags> <user> <server_path> <args>


#:BOOT: TFTP service is provided primarily for booting.  Most sites
#       run this only on machines acting as "boot servers."

tftp            dgram   udp     wait    nobody  /usr/sbin/tcpd  /usr/sbin/in.tftpd /srv/tftp

Listing 1: Our (abbreviated) main config file for “inetd” under which tftpd runs.

The first entry (on the first line) we should pay attention to in Listing 1 refers to update-inetd. This command needs to be run to enable and disable any tftpd services after we make changes to this file.

For example, if you wanted to change something like the directory of where files reside, then edit the last line in our abbreviated config file, altering “/srv/tftp” in this case, and then run the update-inetd command to set that change live.

There’s a nice old-school way of quickly commenting out all of the unencrypted access services with one fell swoop in the /etc/inetd.conf file by using the update-inetd command. It’s described in the manual and looks like this:

# update-inetd --comment-chars '#' --disable login,shell,exec,telnet

As you can see, we’re prepending comments to each line supporting these services to disable them by running this command. There’s are other useful options in the manual if you want to explore:

# man update-inetd

Again, in Listing 1, we can see that these days most people just use the TFTP protocol for boot server scripts. On a LAN, however, tftpd still has its place for other services, too, such as common read-only configs — for example, to reflect changes in NTP servers or other widely used services.

The eagle-eyed among you might have spotted /usr/bin/tcpd being mentioned inside the newly installed, old-timer, config file that is /etc/inetd.conf. This refers to TCP Wrappers, which allow us to control the IP address ranges or domain names that can connect to any service that links in with the library libwrap. It’s an excellent addition to any network-facing service and tftpd is no exception. Incidentally, if you wanted to check if a particular item has been configured to use the functionality provided by TCP Wrappers, you could run a command such as this:

# ldd /usr/sbin/sshd | grep libwrap

libwrap.so.0 => /lib64/libwrap.so.0 (0x00007f07e3066000)

The test above does indeed prove that our OpenSSH Server uses libwrap.so.0. To check another program, such as a mail server, simply replace the /usr/sbin/sshd with the path to the binary file that you wish to query.

To enable TCP Wrappers for your tftpd service you can quickly edit the /etc/hosts.deny file and add the following line:

tftpd: ALL

Then, inside /etc/hosts.allow, you can add a few rules of who can connect to the small file repository being served by your TFTP daemon. An example of an IP address might be:

tftpd: 10.10.10.

Note the trailing dot, which opens up all 254 hosts under the Class C “10.10.10.0” network. Additionally, it’s possible to allow specific hosts by using DNS names such as this example:

tftpd: .workstations.chrisbinnie.tld

Here we allow any workstation host (note the leading dot this time) under our Domain Name to connect without having to put an entry for each. There’s much more information available on the highly recommended TCP Wrappers here:

# man hosts_access

IPtables

If you felt the need to tinker with your existing IPtables scripts (provided courtesy of the excellent kernel-based firewall Netfilter), then you would add a line similar to this one below:

# iptables -I INPUT -p udp --dport 69 -j ACCEPT

This opens up UDP port 69 for inbound traffic as we can see.

Lift Off

If you run the command below to see if anything is listening on UDP port 69, then you can tell whether inetd has fired up tftpd’s daemon already.

# lsof -i :69

COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME

inetd   9571 root    4u  IPv4 139447      0t0  UDP *:tftp

Lo and behold, there is a service listening intently already, without any prompting right after our installation. Take that as a warning to secure your access immediately or stop the service right after the installation if you can’t spend time securing the service as soon as it is installed.

Next time, we’ll wrap up this series with more options to consider along with several examples of moving files around.

Chris Binnie is a Technical Consultant with 20 years of Linux experience and a writer for Linux Magazine and Admin Magazine. His new book Linux Server Security: Hack and Defend teaches you how to launch sophisticated attacks, make your servers invisible and crack complex passwords.

Advance your career in Linux System Administration! Check out the Essentials of System Administration course from The Linux Foundation.

Sir Tim Berners-Lee Receives 2016 ACM A.M. Turing Award

ACM named Sir Tim Berners-Lee, a Professor at Massachusetts Institute of Technology and the University of Oxford, the recipient of the 2016 ACM A.M. Turing Award. Berners-Lee was cited for inventing the World Wide Web, the first web browser, and the fundamental protocols and algorithms allowing the Web to scale. Considered one of the most influential computing innovations in history, the World Wide Web is the primary tool used by billions of people every day to communicate, access information, engage in commerce, and perform many other important activities.

The ACM Turing Award, often referred to as the “Nobel Prize of Computing,” carries a $1 million prize, with financial support provided by Google, Inc. It is named for Alan M. Turing, the British mathematician who articulated the mathematical foundation and limits of computing.

Read more at ACM

High Performance Logging with Apache BookKeeper

Apache BookKeeper is a high-performance and low-latency cloud storage service, originally designed for write ahead logging. Since its original development, BookKeeper has been expanded and is now used by companies including Twitter, Yahoo, Salesforce, Huawei, and EMC.

In their presentation at the recent Vault conference, Venkateswararao Jujjuri (JV) from Salesforce and Sijie Guo from Twitter provided an overview of Apache BookKeeper and showed some production use cases. In this interview, they provide some additional implementation details.

Linux.com: Can you give our readers some background information about Apache BookKeeper? Why was it developed?

JV and Sijie: Apache BookKeeper was originally developed as a sub-project under Apache ZooKeeper. It was designed for high performance and low latency write ahead logging, with strong consistency, replication and strong durability support. It was originally developed for the HA solution of HDFS.

Right now, it has grown beyond its original scope to become a scalable, high throughput and low latency storage service. It is widely used by multiple companies, like Twitter, Yahoo!, Salesforce, Huawei, EMC. There also various projects that have built over BookKeeper, like Apache DistributedLog and Yahoo Pulsar.

Linux.com: How does it work?

JV and Sijie: BookKeeper is a CP system for immutable data (w/ deletes). Hence, the added immutability of its data gives great availability in addition to its consistency and partition tolerance.  It is a thick client scale-out distributed system and makes cap-adds a breeze. Bookkeeper uses ZooKeeper as its metadata store and also consensus engine to manage/maintain the cluster.

The beauty of the design is that interaction with the metadata server is very very minimal. The client talks to the metadata server only during open/create and close times of the ledger, and it doesn’t come in the IO path. This gives greater performance that is less prone to failures.

Linux.com: Are there similar products available? How does it differ?

There are tons of products that offer scale-out storage solutions. But what makes BookKeeper unique is its ability to offer a short-tailed, low-latency, distributed scale-out storage solution. Although this is a CP system, its greater availability makes it almost a C(A)P system. It is an apt storage for immutable data.

Linux.com: Have you encountered challenges in its implementation? If so, how have you addressed them?

JV and Sijie: Yes. There are a lot of interesting implementation details inside Apache BookKeeper. For example, Apache BookKeeper has very low latency with high throughput while still maintain strong consistency and durability.

This is because of various reasons:

First of all, the storage was designed for I/O isolation. It separates the journal disk (which requires large sequential writes and group fsync to persist data) from the ledger disks (which are used for storing indexed data, require fast random reads) physically. So, it avoids the I/O contention between writes and reads, to achieve low latency while we do fsync to ensure durability.

Second, we use a quorum-vote protocol on writing data. Data is written parallel to multiple replicas and the client is waiting for acknowledges from majority. It helps reduce the write latency to avoid impacts from any slow bookies.

Third, at read side, we use a speculative read mechanism on reading data. The speculative mechanism works in this way: The client first issues one read request to one of the replicas, if the read request doesn’t respond with a given time (speculative read timeouts), it will then issue a second read request. The client will wait for responses from both requests. The first responded request will satisfy the read request. If we tune the speculative read timeout to be aligned with 99.9th percentile latency, we will reduce the tail latency.

There are also other interesting aspects in Apache BookKeeper, such as how we ensure consistency, how we do group fsync, etc. Feel free to reach out to us at mailing list: user@bookkeeper.apache.org and dev@bookkeeper.apache.org.

Linux.com: What additional features or further development are planned for Apache BookKeeper?

JV and Sijie: Apache BookKeeper has been successfully used in messaging or streaming area for real-time data. As we grow the project to support more storage use cases, we want to make sure it can also be use a very good long-term storage. We are also working with multiple cluster schedulers (like Mesos and Kubernetes) to make sure it can run easily in different cloud environments. Security is also another big feature coming out in the next release 4.5.0. It will be available soon — around April/May.

Learn first-hand from the largest collection of global Apache communities at ApacheCon 2017 May 16-18 in Miami, Florida. Linux.com readers get $30 off their pass to ApacheCon. Select “attendee” and enter code LINUXRD5. Register now >>  

OPNFV Unveils Danube Release to Bolster Open Source NFV Adoption

OPNFV said its fifth platform release brings development and operations methodologies to the NFV space, and is a “turning point” for organization.

The Linux Foundation’s Open Platform for NFV project launched its fourth platform release, dubbed “Danube,” which the organization said provides key architectural components and improvements designed to foster the adoption of network functions virtualization systems.

OPNFV said the platform, which was released at this week’s Open Networking Summit, brings development and operations methodologies to the NFV space through “collaborative upstream development, integration, deployment and significant continuous integration/continuous development testing automation.”

Read more at RCR Wireless

Blockchain for IoT Extends Beyond Ensuring Security

Blockchain, the technology that made Bitcoin possible, has been getting a lot of attention in the IoT world, often because of its role in security. However, experts and practitioners said the potential of blockchain for IoT is deeper and broader than just keeping the bad guys out.

Ian Hughes, analyst of IoT at 451 Research, sees a role for blockchain that goes deeper, enabling authentication of devices — especially when they are connected infrequently, as the case might be with, say, agricultural systems that may shut down for large parts of the year. Having a blockchain distributed ledger can provide a tidy way to account for and recognize the return of long-lost network participants as trusted members. 

Read more at TechTarget

5 Cool C/C++ App Dev Tools

As compelling as new languages like Rust are for building systems, C and C++ remain fundamental for writing applications that run close to the metal, despite the waxing and waning of their usage statistics.

What’s more, the culture of tools for C/C++ development remains deep and fruitful. Here are five C-related projects — compilers, libraries, and support tools — that caught our eye recently, whether for bolstering existing projects or starting new ones.

Read more at InfoWorld

Builder Pattern vs. Multi-Stage Builds in Docker

This post looks at two new PRs from the Docker project that vastly improve the developer experience for building small images efficiently.

These changes are bleeding edge and are not available in a release yet, but I wanted to test them out.

Docker PR has just been merged to enable multi-stage builds and a second PR opened just after that to improve the UX even further.

Read more at Alexellis.io

Vulnerability Remediation – You Only Have 4 Options

In my previous post, I wrote about a simple process for triaging vulnerabilities across applications. Once you have the issues prioritized, the vulnerability remediation process is pretty straightforward. You don’t have a lot of options; either remediate the issue, ignore it, or apply other measures (compensating controls) to mitigate the risk posed by the vulnerability.

1. Rip and Replace

This is the most common approach taken. Essentially, you are going to fix the problem by “amputating” the vulnerable component and replacing it with a component that fixes the vulnerability (either directly or by using a different open source project).

Read more at BlackDuck

Scalable Microservices with gRPC, Kubernetes, and Docker by Sandeep Dinesh, Google

https://www.youtube.com/watch?v=xsIwYL-N4vI?list=PLfMzBWSH11xYaaHMalNKqcEurBH8LstB8

Together, Kubernetes and gRPC, provide a comprehensive solution to the complexities involved in deploying a massive number of microservices to a cluster.