Diversity Scholarship Series: My Programming Journey – Becoming a Kubernetes Maintainer

By

-

December 16, 2016

I’m Lucas Käldström from Finland. I speak Swedish as my mother tongue in the same manner as 300,000 others in my country do. I just turned 17 and am attending my second year in general upper secondary school. In my spare time, I play soccer, program, go to the gym and read a good book.

screen-shot-2016-11-23-at-9-21-59-am

I’ve always been interested in Math, and am quite good at it as well. So when I was about 13, I started to become interested in programming. I found it interesting because I could command the computer to do nearly anything. I’ve always loved creating things and it was fascinating to see that every change you make can make a difference in a good way. I started creating small programs with VB.NET and C# and about a year later switched to Node.js and web programming (HTML, CSS and JS). At this point I started to feel the power of open source and what it could do. Also, I made myself a Github account in order to be ready if I found a project to contribute to.

In the beginning of May 2015, I first noticed Kubernetes. I got so excited that I could use something Google has designed free of charge! Unfortunately, I did not have any normal Linux hardware I could use at the time. However, I had two Raspberry Pis that I had been tinkering with a little bit. My Bash skills were practically non-existent and most of the time I was scared of typing something into the command line; however, I realized that Raspberry Pi in fact is the ultimate tool to use when teaching Kubernetes to someone with little cloud computing experience as the cluster becomes really practical. You literally get the “hands-on” experience that is so valuable. This later became the main theme for a 163-page master’s thesis paper Martin Jensen and Kasper Nissen wrote. Likewise,Ray Tsang has been travelling a lot with his Raspberry Pi cluster as well, but now I am getting ahead of myself.

After a lot of hacking in May, I got Kubernetes running on my Raspberry Pi, but it was quite pointless as Docker on ARM had a bug that made it impossible to run any containers via Kubernetes. I continued to improve my scripts during the summer, while not playing soccer or doing something else, like swimming! In August of 2015, I tried the same programming with the v1.0.1 Kubernetes release, and I got it working! That was a truly amazing feeling. I quickly started to expand the context to make it more generic, reproducible and faster. In mid-September, I had it working well, and noticed that a Glasgow University group had done the same thing; both of us working beside each other without knowing the other one.

I knew my work could help others, so I quickly published the source I had to the world; it was the right thing to do. I wanted to help more people run, test and learn from a Kubernetes cluster running on small Raspberry Pis… as well as other devices. This project is known as Kubernetes on ARM. After that, I continued to make lots of improvements on it, with feedback from others suddenly! One of the best moments was when someone reported the first issue and showed interest in helping to improve the project. I was part of the open source community!

I wanted to make my work even more widespread and bring it to the core. And so I did. In November-December I started making myself more familiar with the source code, the contributing process, etc.

On December 14, 2015, I got my first Pull Request merged. What a great feeling! I admit it was really small (a removal of 6 chars from a Makefile), but it was a big step personally to realize that the Kubernetes maintainers wanted my contributions. From January-March, I focused on getting the Kubernetes source code to cross-compile ARM binaries and release them automatically. Kubernetes v1.2.0 was the first release that shipped with “official” Google-built ARM binaries.

I then started to focus on getting an official deployment method multiarch-ready. I chose docker-multinode. The result of that work made Kubernetes v1.3.0 ship ARM and ARM 64-bit binaries and corresponding hyperkube images, which made it possible to run a cluster on different architectures with the same documented commands.

In April 2016, I was added to the Kubernetes organization. I couldn’t believe it! I was one out of about 170 at the time. One week later, I became a Kubernetes Maintainer, with a big M! It was totally crazy! I was 16 and got write permission to several repositories! But with great permissions comes great responsibility, and I always have been looking to the projects’ best when reviewing something for instance. In fact, maintainership should be a very important but boring task.

That same month, I noticed that a project called minikube was added to the Kubernetes organization. The repo was empty, just a markdown file, nothing more. I noticed Dan Lorenc started committing to the repo and I thought it would be cool to improve my Go skills by starting a project from scratch, so I started working with him. I became a maintainer for minikube as well, and am still the 6th-highest contributor to that project by commits.

I continued to contribute to Kubernetes during the summer and I worked on minikube until release v0.5.0 was out. Then I switched focus to improving the Kubernetes deployment experience. sig-cluster-lifecycle added me to their group and it turned out to be a great fit for me and my interests. Subsequent Kubernetes work includes:

Wrote a proposal (a design goal) for Kubernetes about how multi-platform clusters should be supported, set up and managed. It got merged in August and is one of the guidelines for how Kubernetes should be improved.

Making Kubernetes easier to install in the sig-cluster-lifecycle group. We have made a tool called kubeadm that makes the setup process much easier than before. This is a crucial effort in my opinion, since right now it is hard for new users to know what deployment solution to use and how they relate to each other. kubeadm will be a building block all higher-level (turn-key) deployment tools can use. This way we are hoping we can somewhat standardize and simplify the Kubernetes deployment story. I’ve also written a document about what we’d like to do in time for v1.6, please check it out if you’re interested.

All in all, I’ve learned a lot thanks to being able to contribute to Kubernetes. And it has been a lot of fun to be able to actually make a difference, which now has led to that I have more than 150 commits to the Kubernetes organization in total. My Kubernetes journey started with a Raspberry Pi, and we don’t know how it ends.

One thing to remember is that I’ve never taken any computing or programming classes. This has been my spare-time hobby. I’ve learned practically everything I can just by doing and participating in the community, which really shows us the power of the internet combined with the will to create something and make a difference.

I am not monetarily paid for my work on the Kubernetes project, but I have gained knowledge, experience, better English communication skills, respect, trust and so on. That is worth more than money right now. Also, I got about five full-time job opportunities during the summer from large worldwide companies without even asking for a job or listing myself as job-searching.

What I’ve really enjoyed while coding on Github is my partial anonymity. My full name has always been there, my email, my location, etc., but I haven’t written that I’m just 17 (or 16 at the time), so people on Github haven’t judged me for being a minor or for not having been to University or for the fact that I’ve never taken a computing class or worked for a big tech company. The community has accepted me for who I am, and respected me.

That’s the power of open source in my opinion. Regardless of who you are when you’re away from the keyboard, you are allowed to join the party and have fun with others like-minded and make a difference together. Diversity is the true strength of open source, and I think diversity scholarships like the one the Cloud Native Computing Foundation provided me to attend CloudNativeCon/KubeCon 2016 in Seattle are a powerful way to make the community even stronger. Thanks!

Apply now for a 2017 diversity scholarship to attend CloudNativeCon Europe, March 29-30, 2017. Applications are due by Feb. 3.

This article originally appeared at Cloud Native Computing Foundation.

How to Make Linux More Trustworthy

By

ArsTechnica

-

December 16, 2016

While the fight against government-mandated software backdoors raged for most of 2016—including the showdown between Apple and the FBI over the San Bernardino shooter’s iPhone, and the UK’s new Investigatory Powers Act, which gives the government the power to demand UK companies backdoor their software to enable mass surveillance—the Core Infrastructure Initiative (CII) has been quietly working to prevent an even more insidious form of backdoor: malicious code inserted during the software build process without a developer’s knowledge or consent.

Such attacks are by no means theoretical. Documents from Snowden’s trove make clear that intelligence agencies are actively working to compromise the software build process on individual developer’s computers. The CIA created a bogus version of XCode, the software used by developers to package applications for Apple devices. Such attacks have been difficult to detect—until now.

Led by the Debian Project, other Linux distros and software projects, including the Tor Browserand Bitcoin, are also working to make their build processes more trustworthy, …

Open Source and the Software Supply Chain

By

OpenSource.com

-

December 16, 2016

Grasping the nuances of hardware supply chains and their management is straightforward—you essentially are tracking moving boxes. Managing something as esoteric as resources for building software with a variety of contributions made by the open source community is more amorphic.

When thinking about open source platforms and supply chains, I thought of the supply chain as a single process, taking existing open source components and producing a single result, namely a product. Since then, I’ve begun to realize that supply chain management defines much of the open source ecosystems today. That is, those who know how to manage and influence the supply chain have a competitive advantage over those who don’t do it as well, or even grasp what it is.

MariaDB ColumnStore Adds Simultaneous Analytics, Transactional Processing

By

The New Stack

-

December 16, 2016

MariaDB has released into general availability ColumnStore 1.0, a storage engine allows users to run analytics and transactional processes simultaneously with a single front end on the MariaDB 10.1 database.

While the open source community pushed back against the licensing model of MariaDB’s MaxScale database proxy, ColumnStore is open source, licensed under GPLv2.

ColumnStore is a fork of InnoDB database engine, the default storage engine for MySQL. Despite the popularity of SQL alternatives, the company notes that every OLTP (online transaction processing) and analytics solution has been building a SQL layer.

Fedora and Ubuntu Zero-Days Show that Hacking Desktop Linux is Now a Thing

By

ArsTechnica

-

December 16, 2016

If you run a mainstream distribution of Linux on a desktop computer, there’s a good chance security researcher Chris Evans can hijack it when you do nothing more than open or even browse a specially crafted music file. And in the event you’re running Chrome on the just-released Fedora 25, his code-execution attack works as a classic drive-by.

The zero-day exploits, which Evans published on Tuesday, are the latest to challenge the popular conceit that Linux, at least in its desktop form, is more immune to the types of attacks that have felled Windows computers for more than a decade and have increasingly snared Macs in recent years.

Top 7 Videos from ApacheCon and Apache Big Data 2016

By

The Linux Foundation

-

December 15, 2016

It’s been two years since The Linux Foundation forged a partnership with the Apache Software Foundation to become the producer of their official ASF events. This year, ApacheCon and Apache Big Data continued to grow and gain momentum as the place to share knowledge, ideas, best practices and creativity with the rest of the Apache open source community.

As 2016 draws to a close, we looked back at some of the highlights from ApacheCon and Apache Big Data and collected the 7 videos from our most-read articles about the events in 2016.

These videos help highlight the good work the open source community accomplished for and with Apache projects this year. We hope they inspire you to participate in the community and present your work again at ApacheCon and Apache Big Data, May 16-18, 2017 in Miami. The deadline to submit proposals is February 11!

Submit an ApacheCon Proposal

Submit an Apache: Big Data Proposal

1. IBM’s Wager on Open Source Is Still Paying Off

When IBM got involved with the Linux open source project in 1998, they were betting that giving their code and time to the community would be a worthwhile investment. Now, 18 years later, IBM is more involved than ever, with more than 62,000 employees trained and expected to contribute to open source projects, according to Todd Moore, Vice President of Open Technology at IBM, speaking at ApacheCon in Vancouver in May.

“It became apparent that open source could be the de facto standards we needed to be the engine to go out and drive things,” Moore said in his keynote. “[The contributions] were bets; we didn’t know how this was going to come out, and we didn’t know if open source would grow, we knew there would be roadblocks and things we’d have to overcome along the way, but it had promise. We thought this would be the way of the future.”

Moore reiterated IBM’s commitment to open source, highlighting projects born at IBM’s developerWorks Open (dWOpen), such as SystemML, Toree, and Quarks, and now in the Apache Incubator.

Read our coverage of Moore’s presentation, and watch the full video below.

https://www.youtube.com/watch?v=MG2iZBLz9g8?list=PLGeM09tlguZTvqV5g7KwFhxDlWi4njK6n

2. Open Source is a Positive-Sum Game, Sam Ramji, Cloud Foundry

As open source software matures and is used by more and more major corporations, it is becoming clear that the enterprise software game has changed. Sam Ramji, CEO of the Cloud Foundry Foundation, believes that open source software is a positive sum game, as reflected in his ApacheCon keynote.

Invoking his love of game theory, Ramji stated emphatically that open source software is a positive-sum game, where the more contributors there are to the common good, the more good there is for everyone. This idea is the opposite of a zero-sum game, where if someone benefits or wins, then another person must suffer, or lose.

Read the full coverage and watch the video below.

https://www.youtube.com/watch?v=qvvwAUZYdNk?list=PLGeM09tlguZTvqV5g7KwFhxDlWi4njK6n

3. Apache Milagro: A New Security System for the Future of the Web

With 25 billion new devices set to hit the Internet by 2025, the need for a better worldwide cryptosystem for securing information is paramount. That’s why the Apache Milagro project is currently incubating at the Apache Software Foundation. It’s a collaboration between MIRACL and Nippon Telegram and Telegraph (NTT), and Brian Spector, MIRACL CEO and Co-Founder, discussed the project in his keynote at ApacheCon in May.

Spector said the project was born in a bar on the back of a napkin after a brainstorm about how one would rebuild Internet security from the ground up. That sounds like a lot of work, but Spector believes it’s absolutely necessary: the future of the Web is going to be very different from the past.

Read the full article and watch the video below.

https://www.youtube.com/watch?v=bIaA7-Eady0?list=PLGeM09tlguZTvqV5g7KwFhxDlWi4njK6n

4. Netflix Uses Open Source Tools for Global Content Expansion

“We measured, we learned, we innovated, and we grew.”

Brian Sullivan, Director of Streaming Data Engineering & Analytics at Netflix, recited this recipe for the streaming video giant’s success several times during his keynote address at the Apache Big Data conference in Vancouver today. It was this mantra, combined with an open source toolkit, that took the stand-alone streaming product from a tiny test launch in Canada to making Netflix a global presence.

Read a summary of the presentation and watch the video, below.

https://www.youtube.com/watch?v=hTfIAWhd3qI?list=PLGeM09tlguZQ3ouijqG4r1YIIZYxCKsLp

5. Spark 2.0 Is Faster, Easier for App Development, and Tackles Streaming Data

It only makes sense that as the community of Spark contributors got bigger the project would get even more ambitious. Spark 2.0 came out with three robust new features, according to Ion Stoica, the founder of Databricks.

“Spark 2.0 is about taking what has worked and what we have learned from the users and making it even better,” Stoica said.

Read our coverage of the keynote and watch the full presentation, below.

https://www.youtube.com/watch?v=9xSz0ppBtFg?list=PLGeM09tlguZQ3ouijqG4r1YIIZYxCKsLp

6. IBM Uses Apache Spark Across Its Products to Help Enterprise Customers

IBM is invested in Spark’s machine-learning capabilities and is contributing back to the project with its work on SystemML, which helps create iterative machine-learning algorithms. The company offers Spark-as-a-service in the cloud, and it’s building it into the next iteration of the Watson analytics platform. Basically anywhere it can, IBM is harnessing the efficient power of Apache Spark.

“We at IBM … have noted the power of Spark, and the other big data technologies that are coming in [from the Apache Software Foundation],” said Luciano Resende, an architect at IBM’s Spark Technology Center.

Read the full article and watch the presentation, below.

https://www.youtube.com/watch?v=YW28qItH_tA?list=PLGeM09tlguZQ3ouijqG4r1YIIZYxCKsLp

7. How eBay Uses Apache Software to Reach Its Big Data Goals

eBay’s ecommerce platform creates a huge amount of data. It has more than 800 million active listings, with 8.8 million new listings each week. There are 162 million active buyers, and 25 million sellers.

“The data is the most important asset that we have,” said Seshu Adunuthula, eBay’s head of analytics infrastructure, during a keynote at Apache Big Data in Vancouver in May. “We don’t have inventory like other ecommerce platforms, what we’re doing is connecting buyers and sellers, and data plays an integral role into how we go about doing this.”

About five years ago, eBay made the conscious choice to go all-in with open source software to build its big data platform and to contribute back to the projects as the platform took shape.

Read the full article about eBay and watch the presentation, below.

https://www.youtube.com/watch?v=wKy9IRG4C2Q?list=PLGeM09tlguZQ3ouijqG4r1YIIZYxCKsLp

Share your knowledge and best practices on the technologies and projects driving the future of open source. Submit a speaking proposal for ApacheCon and Apache Big Data today!

Submit an ApacheCon Proposal

Submit an Apache: Big Data Proposal

Not interested in speaking but want to attend? Linux.com readers can register now for ApacheCon or Apache: Big Data with the discount code, LINUXRD5, for 5% off the registration price.

Hardening the Kernel to Protect Against Attackers

By

Eric Brown

-

December 15, 2016

The task of securing Linux systems is so mind-bogglingly complex and involves so many layers of technology that it can easily overwhelm developers. However, there are some fairly straightforward protections you can use at the very core: the kernel. These hardening techniques help developers guard against the bugs that haven’t yet been detected.

“Hardening is about making bugs more difficult to exploit,” explained Mark Rutland, a kernel developer at ARM Ltd, at the recent Embedded Linux Conference Europe 2016 in Berlin. There will always be dangerous bugs that manage to evade the notice of kernel developers, he added. “We do not yet know which particular bugs exist in the next kernel, and we probably won’t for five years,” he said, referring to Kees Cook’s recent analysis of kernel bug lifetimes.

“We see recurring classes of bugs involving things like dereferencing of null pointers or accessing memory controlled by user space, so we can assume that some of the bugs we don’t know about will fall into these buckets.”

Rutland, who earlier this year warned ELC North America attendees about the hidden dangers of unruly caches, noted that bugs are an unavoidable offshoot of programming. Most are relatively benign, but many cause problems, and some can open dangerous vulnerabilities.

“In the kernel 4.8 merge window we fixed over 500 bugs, many of which were in 4.7 or earlier,” said Rutland, noting that while varied techniques are used today to avoid bugs making it into the kernel, some will inevitably slip through and require later fix-ups. This is “slightly terrifying” given the long lifetime of bugs, which might not be discovered until affected devices are end-of-life.

Some of these bugs have significant security implications. Fortunately, kernel developers have in recent years begun to create hardening features that protect against many of the most common bugs. Rutland implored the audience to make use of these hardening features, noting that many are simple to enable, and their protections are “effectively free,” yet don’t see widespread use.

Rutland went on to discuss several of the main classes of hardening protections that pose the least amount of overhead. Here are some edited quotes about each:

Strict kernel memory permissions – “Historically, the kernel has mapped all memory as readable, writable, and executable…which leads to…being able to modify kernel code or const data, or executing data, all of which…are very useful primitives if you’re an attacker. We can get the MMU to enforce these permissions by…mapping that code as read only or mapping constant data as read only and non- executable. If it’s done in the MMU, it’s effectively free, as the hardware is handling it for us.” (For details, study up on CONFIG_DEBUG_RODATA and CONFIG_DEBUG_SET-MODULE_RONX.)

Stack smashing protection – “Stack smashing attacks work on the principle that stacks contain a return address and other data, as well as local variables. On most architectures, the stack grows downwards, and the buffers grow upwards. If you copy some data to a buffer on the stack, and the data is too large to fit in the buffer, you end up overwriting subsequent data, which happens to include the return address. So if an attacker knows what your stack frame layout will look like, they can control where you will return to, and…branch to any code of their choosing…to launch more advanced attacks. Stack smashing protection guards against this by having the compiler insert a secret value known as a canary between the data and the flow control information.” (For details, see CONFIG_STACKPROTECTOR_REGULAR and CONFIG_STACKPROTECTOR_STRONG.)

User/kernel memory segregation – “Typically the kernel shares an address space with user space in hardware. A pointer can encode an address to either space…using the same load and store instructions. If you accidentally dereference an address…controlled by user space, the hardware won’t notice and will happily give you the value, so if an attacker can convince you to dereference the address…it can be used as the basis for a number of attacks. If an attacker puts a buffer of code in a user space address and then uses a stack smashing exploit to branch to that, they can do whatever they want. The MMU can help by letting us change the page table dynamically, which we use to switch processes. Enabling access temporarily and then disabling access…will catch most of these unintentional user memory accesses or branches.”

Stricter permissions – “Some hardware can…automatically prevent arbitrary code execution from a user space buffer. On ARM we have a feature called privilege execute never (PXN), which…says I never want this page to be executed with kernel privileges. x86 has a similar thing called SMEP. An attacker can still branch to another piece of kernel code, so it doesn’t prevent arbitrary execution, but it limits one case. More recently, MMUs have become able to do this with data accesses as well.”

Rutland advised that it will be “years before we have a reasonable number of protections.” He also noted that the protections are not 100 percent effective, and that “we still have to find and fix bugs.”

In Rutland’s view, Linux systems would be more secure if more of these hardening features were turned on by default, which he said happened in kernel 4.9 with now mandatory, aforementioned DEBUG_RODATA. “Resistance is slowly going away for some of these protections,” he said. “Lots of the complaints about the features not looking like kernel code and doing things wrong are being solved quite quickly. People’s opinions about mainline are changing – there’s agreement that yes, we need to do something here.”

Watch the full video of Rutland’s presentation, “Thwarting Unknown Bugs: Hardening Features in the Mainline Linux Kernel” below:

Embedded Linux Conference + OpenIoT Summit North America will be held on February 21 – 23, 2017 in Portland, Oregon. Check out over 130 sessions on the Linux kernel, embedded development & systems, and the latest on the open Internet of Things.

Linux.com readers can register now with the discount code, LINUXRD5, for 5% off the attendee registration price. Register now>>

Building an Email Server on Ubuntu Linux, Part 3

By

Carla Schroder

-

December 15, 2016

Welcome back, me hearty Linux syadmins! In part 1 and part 2 of this series, we learned to how to put Postfix and Dovecot together to make a nice IMAP and POP3 mail server. Now we will learn to make virtual users so that we can manage all of our users in Dovecot.

Sorry, No SSL. Yet.

I know I promised to show you how to set up a proper SSL-protected server. Unfortunately, I underestimated how large that topic is. So, I will realio trulio write a comprehensive how-to by next month.

For today, in this final part of this series, we’ll go into detail on how to set up virtual users and mailboxes in Dovecot and Postfix. It’s a bit weird to wrap your mind around, so the following examples are as simple as I can make them. We’ll use plain flat files and plain-text authentication. You have the options of using database back ends and nice strong forms of encrypted authentication; see the links at the end for more information on these.

Virtual Users

You want virtual users on your email server and not Linux system users. Using Linux system users does not scale, and it exposes their logins, and your Linux server, to unnecessary risk. Setting up virtual users requires editing configuration files in both Postfix and Dovecot. We’ll start with Postfix. First, we’ll start with a clean, simplified /etc/postfix/main.cf. Move your original main.cf out of the way and create a new clean one with these contents:


compatibility_level=2
smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu/GNU)
biff = no
append_dot_mydomain = no

myhostname = localhost
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = $myhostname
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 192.168.0.0/24
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all

virtual_mailbox_domains = /etc/postfix/vhosts.txt
virtual_mailbox_base = /home/vmail
virtual_mailbox_maps = hash:/etc/postfix/vmaps.txt
virtual_minimum_uid = 1000
virtual_uid_maps = static:5000
virtual_gid_maps = static:5000
virtual_transport = lmtp:unix:private/dovecot-lmtp0

You may copy this exactly, except for the 192.168.0.0/24 parameter for mynetworks, as this should reflect your own local subnet.

Next, create the user and group vmail, which will own your virtual mailboxes. The virtual mailboxes are stored in vmail's home directory.


$ sudo groupadd -g 5000 vmail
$ sudo useradd -m -u 5000 -g 5000 -s /bin/bash vmail

Then reload the Postfix configurations:


$ sudo postfix reload
[sudo] password for carla: 
postfix/postfix-script: refreshing the Postfix mail system

Dovecot Virtual Users

We’ll use Dovecot’s lmtp protocol to connect it to Postfix. You probably need to install it:


$ sudo apt-get install dovecot-lmtpd

The last line in our example main.cf references lmtp. Copy this example /etc/dovecot/dovecot.conf, replacing your existing file. Again, we are using just this single file, rather than calling the files in /etc/dovecot/conf.d.


protocols = imap pop3 lmtp
log_path = /var/log/dovecot.log
info_log_path = /var/log/dovecot-info.log
ssl = no
disable_plaintext_auth = no
mail_location = maildir:~/.Mail
pop3_uidl_format = %g
auth_verbose = yes
auth_mechanisms = plain

passdb {
  driver = passwd-file
  args = /etc/dovecot/passwd
}

userdb {
  driver = static
  args = uid=vmail gid=vmail home=/home/vmail/studio/%u
}

service lmtp {
 unix_listener /var/spool/postfix/private/dovecot-lmtp {
   group = postfix
   mode = 0600
   user = postfix
  }
}

protocol lmtp {
  postmaster_address = postmaster@studio
}
service lmtp {
  user = vmail
}

At last, you can create the file that holds your users and passwords, /etc/dovecot/passwd. For simple plain text authorization we need only our users’ full email addresses and passwords:


alrac@studio:{PLAIN}password
layla@studio:{PLAIN}password
fred@studio:{PLAIN}password
molly@studio:{PLAIN}password
benny@studio:{PLAIN}password

The Dovecot virtual users are independent of the Postfix virtual users, so you will manage your users in Dovecot. Save all of your changes and restart Postfix and Dovecot:


$ sudo service postfix restart
$ sudo service dovecot restart

Now let’s use good old telnet to see if Dovecot is set up correctly.


$ telnet studio 110
Trying 127.0.1.1...
Connected to studio.
Escape character is '^]'.
+OK Dovecot ready.
user molly@studio
+OK
pass password
+OK Logged in.
quit
+OK Logging out.
Connection closed by foreign host.

So far so good! Now let’s send some test messages to our users with the mail command. Make sure to use the whole user’s email address and not just the username.


$ mail benny@studio
Subject: hello and welcome!
Please enjoy your new mail account!
.

The period on the last line sends your message. Let’s see if it landed in the correct mailbox.


$ sudo ls -al /home/vmail/studio/benny@studio/.Mail/new
total 16
drwx------ 2 vmail vmail 4096 Dec 14 12:39 .
drwx------ 5 vmail vmail 4096 Dec 14 12:39 ..
-rw------- 1 vmail vmail  525 Dec 14 12:39 1481747995.M696591P5790.studio,S=525,W=540

And there it is. It is a plain text file that we can read:

$ less 1481747995.M696591P5790.studio,S=525,W=540
Return-Path: <carla@localhost>
Delivered-To: benny@studio
Received: from localhost
        by studio (Dovecot) with LMTP id V01ZKRuuUVieFgAABiesew
        for <benny@studio>; Wed, 14 Dec 2016 12:39:55 -0800
Received: by localhost (Postfix, from userid 1000)
        id 9FD9CA1F58; Wed, 14 Dec 2016 12:39:55 -0800 (PST)
Date: Wed, 14 Dec 2016 12:39:55 -0800
To: benny@studio
Subject: hello and welcome!
User-Agent: s-nail v14.8.6
Message-Id: <20161214203955.9FD9CA1F58@localhost>
From: carla@localhost (carla)

Please enjoy your new mail account!

You could also use telnet for testing, as in the previous segments of this series, and set up accounts in your favorite mail client, such as Thunderbird, Claws-Mail, or KMail.

Troubleshooting

When things don’t work, check your logfiles (see the configuration examples), and run journalctl -xe. This should give you all the information you need to spot typos, uninstalled packages, and nice search terms for Google.

What Next?

Assuming your LAN name services are correctly configured, you now have a nice usable LAN mail server. Obviously, sending messages in plain text is not optimal, and an absolute no-no for Internet mail. See Dovecot SSL configuration and Postfix TLS Support. VirtualUserFlatFilesPostfix covers TLS and database back ends. And watch for my upcoming SSL how-to. Really.

Advance your career in system administration! Check out the Essentials of System Administration course from The Linux Foundation.

Docker Open Sources Critical Infrastructure Component

By

TechCrunch

-

December 15, 2016

Docker announced today that it was open sourcing containerd (pronounced Container D), making a key infrastructure piece of its container platform available for anyone to work on.

Containerd, which acts as the core container runtime engine, is a component within Docker that provides “users with an open, stable and extensible base for building non-Docker products and container solutions,” according to the company. Leading cloud providers have signed on to work on it including Alibaba, AWS, Google, IBM and Microsoft.

Data Wrangling at Slack

By

Slack Engineering Blog

-

December 15, 2016

For a company like Slack that strives to be as data-driven as possible, understanding how our users use our product is essential.

The Data Engineering team at Slack works to provide an ecosystem to help people in the company quickly and easily answer questions about usage, so they can make better and data informed decisions: “Based on a team’s activity within its first week, what is the probability that it will upgrade to a paid team?” or “What is the performance impact of the newest release of the desktop app?”

The Dream

We knew when we started building this system that we would need flexibility in choosing the tools to process and analyze our data. Sometimes the questions being asked involve a small amount of data and we want a fast, interactive way to explore the results. Other times we are running large aggregations across longer time series and we need a system that can handle the sheer quantity of data and help distribute the computation across a cluster. Each of our tools would be optimized for a specific use case, and they all needed to work together as an integrated system.