Hear about Linux development directly from Linus Torvalds in this video from our archives.
Linus Torvalds took to the stage in China for the first time Monday at LinuxCon + ContainerCon + CloudOpen China 2017 in Beijing. In front of a crowd of nearly 2,000, Torvalds spoke with VMware Head of Open Source Dirk Hohndel in one of their famous “fireside chats” about what motivates and surprises him and how aspiring open source developers can get started. Here are some highlights of their talk.
What’s surprising about Linux development
“What I find interesting is code that I thought was stable continually gets improved. There are things we haven’t touched for many years, then someone comes along and improves them or makes bug reports in something I thought no one used. We have new hardware, new features that are developed, but after 25 years, we still have old, very basic things that people care about and still improve.”
What motivates him
“I really like what I’m doing. I like waking up and having a job that is technically interesting and challenging without being too stressful so I can do it for long stretches; something where I feel I am making a real difference and doing something meaningful not just for me.”
“I occasionally have taken breaks from my job. The 2-3 weeks I worked on Git to get that started for example. But every time I take a longer break, I get bored. When I go diving for a week, I look forward to getting back. I never had the feeling that I need to take a longer break.”
The future of Linux leadership
“Our processes have not only worked for 25 years, we still have a very strong maintainer group. We complain that we don’t have enough maintainers – which is true, we only have tens of top maintainers who do the daily work of merging stuff. That’s a strong team for an open source project. And as these maintainers get older and fatter, we have new people coming in. It takes years to go from a new developer to a top maintainer, so I don’t feel that we should necessarily worry about the process and Linux for the next 20 years.”
Will Linux be replaced
“Maybe some new aggressive project will come along and show they can do what we do better, but I don’t worry about that. There have been lots of very successful forks of Linux. What makes people not think of them as forks is that they are harmonious. If someone says they want to do this and change everything and make the kernel so much better, my feeling is do it, prove yourself. I may think it’s a bad idea, but you can prove me wrong.”
Thoughts on Git
“I’m very surprised about how widely Git has spread. I’m pleased obviously, and it validates my notion of doing distributed development. At the same time, looking at most source control versions, it tends to be a huge slog and difficult to introduce a new software control version. I expected it to be limited mostly to the kernel — as it’s tailored to what we do.”
“For the first 3 to 4 years, the complaint about Git was it was so different and hard to use. About 5 years ago something changed. Enough projects and developers had started using Git that it wasn’t different anymore; it was what people were used to. They started taking advantage of the development model and the feeling of security that using Git meant nothing would be corrupted or lost.”
“In certain circles, Git is more well known than Linux. Linux is often hidden – on an Android phone you’re running Linux, but you don’t think about it. With Git, you know you are using Git.”
Forking Linux
“When I sat down and wrote Git, a prime principle was that you should be able to fork and go off on your own and do something on your own. If you have forks that are friendly — the type that prove me wrong and do something interesting that improves the kernel — in that situation, someone can come back and say they actually improved the kernel and there are no bad feelings. I’ll take your improved code and merge it back. That’s why you should encourage forks. You also want to make it easy to take back the good ones.”
How to get started as an open source developer
“For me, I was always self-motivated and knew what I wanted to do. I was never told what I should look at doing. I’m not sure my example is the right thing for people to follow. There are a ton of open source projects and, if you are a beginning programmer, find something you’re interested in that you can follow for more than just a few weeks. Get to know the code so well that you get to the point where you are an expert on a code piece. It doesn’t need to be the whole project. No one is an expert on the whole kernel, but you can know an area well.
If you can be part of a community and set up patches, it’s not just about the coding, but about the social aspect of open source. You make connections and improve yourself as a programmer. You are basically showing off – I made these improvements, I’m capable of going far in my community or job. You’ll have to spend a certain amount of time to learn a project, but there’s a huge upside — not just from a career aspect, but having an amazing project in your life.”
With IBM providing its POWER9 processors, Nvidia contributing its Volta V100 GPUs, Mellanox bringing its Infiniband into play, and Red Hat supplying Red Hat Enterprise OS, the level of inter-vendor collaboration has reached something of an all-time high and an amazing new supercomputer is now ready for business.
Devuan is a fork of Debian that eschews the Red Hat-developed systemd init system in favor of alternatives such as sysvinit, among others. Unlike the Mir vs. Wayland controversy, the use of systemd has impacted enterprise servers, which have highly customized init scripts that are challenging to reimplement for systemd-powered systems, or otherwise break across upgrades.
Devuan, pronounced “dev one,” is available for 32- and 64-bit PCs, with specialized ARM images available for certain Chromebooks, as well as the MeeGo-era Nokia N9, N900, and N950 phones, and the Motorola Droid 4. It’s also available for single-board computers including the Raspberry Pi series, ODROID XU and XU4, BeagleBone Black, and Allwinner-powered boards with mainline U-Boot and Linux support, including variants of the Banana Pi and Orange Pi products. Current Debian Jessie and Stretch users can migrate directly to Devuan without needing to start from a fresh installation.
Sonar was developed for and by Facebook engineers to help them manage the social network, including the implementation of new features, bug hunting, and performance optimization.
Now, Sonar is being released to the open source community in the hopes of giving programmers a tool for the acceleration of app development and deployment. … Made up of a desktop client and mobile SDK, Sonar can be used by developers to inspect app layouts — whether or not the apps were built with standard Android/iOS views or Litho/ComponentKit components — as well as inspect both logs and network traffic.
I have one of these Compute Sticks (Figure 1) and use it as an all-purpose server. It is inconspicuous and silent and, as it is built around an x86 architecture, I don’t have problems getting it to work with drivers for my printer, and that’s what it does most days: it interfaces with the shared printer and scanner in my living room.
An Intel ComputeStick. Euro coin for size.
Most of the time it is idle, especially when we are out, so I thought it would be good idea to use it as a surveillance system. The device doesn’t come with its own camera, and it wouldn’t need to be spying all the time. I also didn’t want to have to start the image capturing by hand because this would mean having to log into the Stick using SSH and fire up the process by writing commands in the shell before rushing out the door.
So I thought that the thing to do would be to grab a USB webcam and have the surveillance system fire up automatically just by plugging it in. Bonus points if the surveillance system fired up also after the Stick rebooted, and it found that the camera was connected.
In prior installments, we saw that systemd services can be started or stopped by hand or when certain conditions are met. Those conditions are not limited to when the OS reaches a certain state in the boot up or powerdown sequence but can also be when you plug in new hardware or when things change in the filesystem. You do that by combining a Udev rule with a systemd service.
Hotplugging with Udev
Udev rules live in the /etc/udev/rules directory and are usually a single line containing conditions and assignments that lead to an action.
That was a bit cryptic. Let’s try again:
Typically, in a Udev rule, you tell systemd what to look for when a device is connected. For example, you may want to check if the make and model of a device you just plugged in correspond to the make and model of the device you are telling Udev to wait for. Those are the conditions mentioned earlier.
Then you may want to change some stuff so you can use the device easily later. An example of that would be to change the read and write permissions to a device: if you plug in a USB printer, you’re going to want users to be able to read information from the printer (the user’s printing app would want to know the model, make, and whether it is ready to receive print jobs or not) and write to it, that is, send stuff to print. Changing the read and write permissions for a device is done using one of the assignments you read about earlier.
Finally, you will probably want the system to do something when the conditions mentioned above are met, like start a backup application to copy important files when a certain external hard disk drive is plugged in. That is an example of an action mentioned above.
shows the conditions that the device has to meet before doing any of the other stuff you want the system to do. The device has to be added (ACTION=="add") to the machine, it has to be integrated into the video4linux subsystem. To make sure the rule is applied only when the correct device is plugged in, you have to make sure Udev correctly identifies the manufacturer (ATTRS{idVendor}=="03f0") and a model (ATTRS{idProduct}=="e207") of the device.
In this case, we’re talking about this device (Figure 2):
The HP webcam used in this experiment.
Notice how you use == to indicate that these are a logical operation. You would read the above snippet of the rule like this:
if the device is added and the device controlled by the video4linux subsystem
and the manufacturer of the device is 03f0 and the model is e207, then...
But where do you get all this information? Where do you find the action that triggers the event, the manufacturer, model, and so on? You will probably have to use several sources. The IdVendor and idProduct you can get by plugging the webcam into your machine and running lsusb:
lsusb
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 003: ID 03f0:e207 Hewlett-Packard
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 04f2:b1bb Chicony Electronics Co., Ltd
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
The webcam I’m using is made by HP, and you can only see one HP device in the list above. The ID gives you the manufacturer and the model numbers separated by a colon (:). If you have more than one device by the same manufacturer and not sure which is which, unplug the webcam, run lsusb again and check what’s missing.
OR…
Unplug the webcam, wait a few seconds, run the command udevadmin monitor --environment and then plug the webcam back in again. When you do that with the HP webcam, you get:
That may look like a lot to process, but, check this out: the ACTION field early in the list tells you what event just happened, i.e., that a device got added to the system. You can also see the name of the device spelled out on several of the lines, so you can be pretty sure that it is the device you are looking for. The output also shows the manufacturer’s ID number (ID_VENDOR_ID=03f0) and the model number (ID_VENDOR_ID=03f0).
This gives you three of the four values the condition part of the rule needs. You may be tempted to think that it a gives you the fourth, too, because there is also a line that says:
SUBSYSTEM=input
Be careful! Although it is true that a USB webcam is a device that provides input (as does a keyboard and a mouse), it is also belongs to the usb subsystem, and several others. This means that your webcam gets added to several subsystems and looks like several devices. If you pick the wrong subsystem, your rule may not work as you want it to, or, indeed, at all.
So, the third thing you have to check is all the subsystems the webcam has got added to and pick the correct one. To do that, unplug your webcam again and run:
ls /dev/video*
This will show you all the video devices connected to the machine. If you are using a laptop, most come with a built-in webcam and it will probably show up as /dev/video0. Plug your webcam back in and run ls /dev/video* again.
Now you should see one more video device (probably /dev/video1).
Now you can find out all the subsystems it belongs to by running udevadm info -a /dev/video1:
udevadm info -a /dev/video1
Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device.
looking at device '/devices/pci0000:00/0000:00:1c.3/0000:04:00.0
/usb3/3-1/3-1:1.0/video4linux/video1':
KERNEL=="video1"
SUBSYSTEM=="video4linux"
DRIVER==""
ATTR{dev_debug}=="0"
ATTR{index}=="0"
ATTR{name}=="HP Webcam HD 2300: HP Webcam HD"
[etc...]
The output goes on for quite a while, but what you’re interested is right at the beginning: SUBSYSTEM=="video4linux". This is a line you can literally copy and paste right into your rule. The rest of the output (not shown for brevity) gives you a couple more nuggets, like the manufacturer and mode IDs, again in a format you can copy and paste into your rule.
Now you have a way of identifying the device and what event should trigger the action univocally, it is time to tinker with the device.
The next section in the rule, SYMLINK+="mywebcam", TAG+="systemd", MODE="0666" tells Udev to do three things: First, you want to create symbolic link from the device to (e.g. /dev/video1) to /dev/mywebcam. This is because you cannot predict what the system is going to call the device by default. When you have an in-built webcam and you hotplug a new one, the in-built webcam will usually be /dev/video0 while the external one will become /dev/video1. However, if you boot your computer with the external USB webcam plugged in, that could be reversed and the internal webcam can become /dev/video1 and the external one /dev/video0. What this is telling you is that, although your image-capturing script (which you will see later on) always needs to point to the external webcam device, you can’t rely on it being /dev/video0 or /dev/video1. To solve this problem, you tell Udev to create a symbolic link which will never change in the moment the device is added to the video4linux subsystem and you will make your script point to that.
The second thing you do is add "systemd" to the list of Udev tags associated with this rule. This tells Udev that the action that the rule will trigger will be managed by systemd, that is, it will be some sort of systemd service.
Notice how in both cases you use += operator. This adds the value to a list, which means you can add more than one value to SYMLINK and TAG.
The MODE values, on the other hand, can only contain one value (hence you use the simple = assignment operator). What MODE does is tell Udev who can read from or write to the device. If you are familiar with chmod (and, if you are reading this, you should be), you will also be familiar of how you can express permissions using numbers. That is what this is: 0666 means “give read and write privileges to the device to everybody“.
At last, ENV{SYSTEMD_WANTS}="webcam.service" tells Udev what systemd service to run.
Save this rule into file called 90-webcam.rules (or something like that) in /etc/udev/rules.d and you can load it either by rebooting your machine, or by running:
sudo udevadm control --reload-rules && udevadm trigger
Service at Last
The service the Udev rule triggers is ridiculously simple:
Basically, it just runs the checkimage.sh script stored in your personal bin/ and pushes it the background. This is something you saw how to do in prior installments. It may seem something little, but just because it is called by a Udev rule, you have just created a special kind of systemd unit called a device unit. Congratulations.
As for the checkimage.sh script webcam.service calls, there are several ways of grabbing an image from a webcam and comparing it to a prior one to check for changes (which is what checkimage.sh does), but this is how I did it:
#!/bin/bash
# This is the checkimage.sh script
mplayer -vo png -frames 1 tv:// -tv driver=v4l2:width=640:height=480:device=
/dev/mywebcam &>/dev/null
mv 00000001.png /home/[user name]/monitor/monitor.png
while true
do
mplayer -vo png -frames 1 tv:// -tv driver=v4l2:width=640:height=480:device=/dev/mywebcam &>/dev/null
mv 00000001.png /home/[user name]/monitor/temp.png
imagediff=`compare -metric mae /home/[user name]/monitor/monitor.png /home/[user name]
/monitor/temp.png /home/[user name]/monitor/diff.png 2>&1 > /dev/null | cut -f 1 -d " "`
if [ `echo "$imagediff > 700.0" | bc` -eq 1 ]
then
mv /home/[user name]/monitor/temp.png /home/[user name]/monitor/monitor.png
fi
sleep 0.5
done
Start by using MPlayer to grab a frame (00000001.png) from the webcam. Notice how we point mplayer to the mywebcam symbolic link we created in our Udev rule, instead of to video0 or video1. Then you transfer the image to the monitor/ directory in your home directory. Then run an infinite loop that does the same thing again and again, but also uses Image Magick’s compare tool to see if there any differences between the last image captured and the one that is already in the monitor/ directory.
If the images are different, it means something has moved within the webcam’s frame. The script overwrites the original image with the new image and continues comparing waiting for some more movement.
Plugged
With all the bits and pieces in place, when you plug your webcam in, your Udev rule will be triggered and will start the webcam.service. The webcam.service will execute checkimage.sh in the background, and checkimage.sh will start taking pictures every half a second. You will know because your webcam’s LED will start flashing indicating every time it takes a snap.
As always, if something goes wrong, run
systemctl status webcam.service
to check what your service and script are up to.
Coming up
You may be wondering: Why overwrite the original image? Surely you would want to see what’s going on if the system detects any movement, right? You would be right, but as you will see in the next installment, leaving things as they are and processing the images using yet another type of systemd unit makes things nice, clean and easy.
Just wait and see.
Learn more about Linux through the free “Introduction to Linux” course from The Linux Foundation and edX.
The LF Deep Learning Foundation, a community umbrella project of The Linux Foundation with the mission of supporting artificial intelligence, machine learning and deep learning open source projects, is working to build a self-sustaining ecosystem of projects. Having a clear roadmap for how to contribute projects is a first step. Contributed projects operate under their own technical governance with collaboration resources allocated and provided by the LF Deep Learning Foundation’s Governing Board. Membership in the LF Deep Learning Foundation is not required to propose a project contribution.
Share your expertise! Submit your proposal to speak at ELC + OpenIoT Summit Europe by July 1.
For the past 13 years, Embedded Linux Conference (ELC) has been the premier vendor-neutral technical conference for companies and developers using Linux in embedded products. ELC has become the preeminent space for product vendors as well as kernel and systems developers to collaborate with user-space developers – the people building applications on embedded Linux.
If you’re just getting started with a squad model, you may not be sure what roles you’ll need for your team to function smoothly. Our squad model in the IBM Digital Business Group is based on the Spotify Squad framework. At a high level, a squad is a small, cross-functional team that has autonomy to deliver on their squad mission. The squad missions and cross-squad priorities are set at an organizational level. Then within each squad, they decide “what to build, how to build it, and how to work together while building it.
“The key takeaway here is not that our way of assigning responsibilities to roles is the best or only way to do it. In fact, we have some variation between the squads in our own organization. The most important thing is that it’s crystal clear to everyone involved who is responsible for what. Perhaps you’ll see something on this list of responsibilities that you were missing before. So, let’s dive in with the first role.
The R Consortium Infrastructure Steering Committee (ISC) is exploring the benefits of recommending that R package authors, contributors, and maintainers adopt the Linux Foundation (LF)Core Infrastructure Initiative (CII)Best Practices Badge. This badge provides a means for Free/Libre and Open Source Software (FLOSS) projects to highlight to what extent package authors follow best software practices, while enabling individuals and enterprises to assess quickly a package’s strengths and weaknesses across a range of dimensions. The CII Best Practices Badge Program is a voluntary, self-certification, at no cost. An easy to use web application guides users in the process.
More companies are using their security logs to detect malicious incidents. Many of them are collecting too much log data—often billions of events. They brag about the size of their log storage drive arrays. Are they measured in terabytes or petabytes?
A mountain of data doesn’t give them as much useful information as they would like. Sometimes less is more. If you get so many alerts that you can’t adequately respond to them all, then something needs to change. A centralized log management system can help. This quick intro to logging and expert recommendations help new system admins understand what they should be doing to get the most out of security logs.
Security event logging basics
One of the best guides to security logging is the National Instituted of Standards & Technology (NIST) Special Publication 800-92, Guide to Computer Security Log Management. Although it’s a bit dated, written in 2006, it still covers the basics of security log management well.
It places security log generators into three categories: operating system, application, or security-specific software (e.g., firewalls or intrusion detection systems [IDS]). Most computers have dozens of logs. …
Unix-style systems usually have syslog enabled by default, and you can configure the detail level. Many other application and security log files are disabled by default, but you can enable each individually with a single command line.
Each network device and security application and device will generate its own logs. Altogether, a system administrator will have hundreds of log files to choose from. A typical end-user system can generate thousands to tens of thousands of events per day. A server can easily generate hundreds of thousands to millions of events a day.
