This is the lightly-edited transcript of a NewsForge IM chat with Doug Abbott, a leading real-time Linux expert and proponent.NewsForge: How can Linux compete with all the specialized embedded operating systems out there? Aren't some of them better-designed for real-time operations?
Abbott: Yes, the "traditional" embedded OSes are indeed better designed for real-time. Linux "competes" because it's "free" and you get the source code.
Now you have to ask the question, "How much real-time do you really need?"
NewsForge: And the answer is...
Abbott: Well, many systems have aspects of both "hard" real-time and "soft" real-time. If you're controlling a robot for example, the task of monitoring arm movement is definitely hard real-time. The task that displays the arm movement for the operator isn't. The strategy behind RTLinux and its variant RTAI is to do the non-real-time stuff with Linux and the real-time stuff with a small deterministic kernel.
NewsForge: A kernel custom-written for the specific device and application?
Abbott: No. Maybe I'm getting ahead of myself here. The RTLinux strategy is called "interrupt abstraction." Linux runs as the lowest priority task, the idle task if you will, under a small real-time kernel. The real-time stuff then runs as higher priority tasks under this kernel.
The RT kernel takes over interrupt handling (hence the name "interrupt abstraction"). Interrupts intended for Linux are passed to it when Linux has interrupts enabled. Interrupts intended for the RT tasks are passed directly to them.
NewsForge: So you're using Linux - or at least some part of the Linux kernel - right down to the arm movement?
Abbott: No, the arm movement is handled by an RT task running directly under the RT kernel. RT tasks can't use kernel services because the kernel is not preemptible. That's why standard Linux isn't real-time. On the other hand, you're using Linux and X Windows to graph the arm movement.
NewsForge: Can your point us at examples of RT kernels you use/recommend?
Abbott: : Well, in the context of RTLinux and RTAI, the kernel is "built in." It's part of the implementation.
Have you seen the RTLinux HOWTO?
Abbott: I have now. I just brought it up.
NewsForge: The thing is, very few people who haven't directly dealt with real-time processes have any understanding at all of how they work and the software behind them.
Abbott: I agree. Multitasking and interrupt-driven programming are not easy concepts to grasp initially.
NewsForge: I've heard Linux is starting to penetrate process control and other industries run real-time (or nearly real-time) computers in plant environments. Is this penetration getting strong enough that young programmers should look into specializing in this area?
Abbott: Any "young programmer" should be exposed to the concepts of embedded and real-time programming whether it be in the Linux environment or some other environment and whether or not it's process control. Embedded programming used to be a rather obscure niche of computer science. Now that computers are everywhere, so are embedded programmers.
NewsForge: Any specific book or course recommendations?
Abbott: You mean in addition to my book 8-)?
A couple others I might suggest are "An Embedded Software Primer" by David Simon,
"MicroC/OS-II, The Real-time Kernel" by Jean LaBrosse, and also "Embedded Systems Building Blocks" by Labrosse. The embedded software bookshelf seems to be getting longer these days. There are probably other good ones. These are just ones I'm familiar with.
Abbott: I should also add that good embedded programmers need to have some exposure to hardware. As for coursework, historically there hasn't been much at the university level but I sense that that has changed over the past couple of years. I'm aware of several universities that now have courses dealing with embedded programming whether it be with Linux or something like MicroC/OS
NewsForge: What about *your* courses? :)
Abbott: Ah. I teach a pair of 3-day seminars on real-time and embedded programming. The first one deals with the fundamentals of the multi-taksing paradigm and how you use that paradigm to structure reliable and understandable interrupt-driven code. That course uses MicroC/OS as the basis. The other one looks at Linux as a basis for embedded and real-time programming.
I think of these two as looking at opposite ends of the embedded spectrum. You're not going to put Linux on an 8051 embedded in a microwave oven. By the same token, MicroC/OS doesn't have the facilities to power a PDA or a settop box.
NewsForge: Now back specifically to Linux... is licensing really enough of a cost to make a difference here?
Abbott: Yes, I think it can be. Wind River charges on the order of $20k per seat plus royalties for VxWorks. But we should also recognize that Linux isn't exactly "free." I came across a great quote from Linus a couple years ago: "Software is like sex. It's better when it's free." Well, we all know that sex is never free. There's always a catch. And I contend that software isn't free either. You can pay someone like Wind River up front and get a solid, well-documented, well-supported product, or you can download Linux from the net and you're on your own. If you're not already a Unix guru, the Linux learning curve is very steep.
NewsForge: What about the "in-between" step of hiring skilled Linux people?
Abbott: Sure. I guess the point is that to use Linux you probably need more in-house OS expertise than you might need to use a COTS (Commercial Off The Shelf) solution. The ultimate payoff for using Linux is much greater transparency, because you have the source code, and access to a large community of open source developers.
NewsForge: What about the ability to modify/customize? Is that a big deal, assuming you have or can buy the expertise needed?
Abbott: Personally, I would be very cautious about modifying the kernel itself. But access to the source code makes it somewhat easier to write (or modify) device drivers, and various kernel "hooks" let you add capability without actually hacking the kernel.
NewsForge: Ah. And device drivers... I know this calls for a subjective judgement, but are they easier or harder to write for Linux or for, say, Wind River's system or QNX?
Abbott: Well, again, access to the source code makes Linux device drivers quite transparent. You can see exactly how they work, how they interact with the kernel and you've got a plethora of examples to use as starting points for your own drivers. Wind River and QNX just give you the APIs and a few examples. I've never had to write a VxWorks driver but I'd guess it would be easier in Linux.
NewsForge: Anything I should be asking but I'm too stupid to ask? :)
Abbott: How about, "Isn't Linux 'too big' to embed?"
Abbott: Because you can custom configure the kernel, you can take out a lot of functionality that you don't need in your specific application. It's not difficult to put a reasonably functional Linux kernel and application code on a 1.44 Mbyte diskette. 8 Mbytes of RAM is about the minumum to do anything useful with Linux.
There are a number of commercial distributions of embedded Linux with stripped down kernels, nice configuration tools and other goodies.
That is to say, distributions that are specifically oriented toward embedded applications.
NewsForge: Is Linux increasing its share of the real-time OS marketplace?
Abbott: Yes. It's not necessarily getting a bigger share of the "real-time" OS marketplace, but it's definitely becoming a major factor in the embedded space. Keep in mind that not all embedded devices have hard real-time requirements. Many devices can get along just fine with plain old Linux.