San Diego -- Mainframe programmers and sysadmins get more than just sales pitches and informational talks at SHARE this week -- they also get real-world training. I sat in on a number of educational sessions and hands-on labs at the conference, taught by engineers from IBM, Novell, and independent software vendors.
Planning, diagnosing, and comparing
Novell's Mark Post, maintainer of the Linux on zSeries resource site linuxvm.org, lectured on planning for mainframe Linux installations. The talk addressed several technical decisions, such as which system architectures are best suited for which applications, and the relative advantages of different networking and storage hardware. But it also dealt with management decisions like determining how to allocate responsibility for supporting Linux virtual servers on a mainframe -- a question Post described as generating "rancorous discussion in most places" that traditionally maintain separate mainframe and server support staff, an observation that elicted laughter from the crowd.
In "Help! My (Virtual) Penguin Is Sick!" Phil Smith of Velocity Software discussed the basics of diagnosing and debugging problems on Linux virtual machines (or "guests," as they are known in mainframe virtualization circles) on IBM's z/VM operating system. He led off with a slide that read "Applications people worry about how it will work, systems people worry about how it will fail." For the z/VM crowd, he continued, the ailment is always "my Linux guest isn't responding" -- and the cause could be any combination of networking problems, Linux problems, or z/VM problems.
The remainder of the talk outlined practical tasks: establishing baseline characteristics when things are running correctly, checklists for diagnosing misbehavior, and so on. But Smith also took time to describe the basics of systems administration in Linux -- how kernel logging works, and the different parts of the filesystem and how they are traditonally arranged on disk partitions.
Along the way he described some of Linux's shortcomings from z/VM's point of view; apparently Linux systems are quite greedy with system resources (they use all the memory they can grab and don't let go), and apparently cron wreaks havoc on z/VM by waking up the system regularly to check for scheduled jobs even when most of the time there are none.
Post and Smith both took occasion to comment on the differences between Linux and mainframes OSes, which made for another interesting glimpse into the mainframe world. Post held up the /proc and /sys virtual filesystems as a welcome innovation, remarking on the ease with which they permitted system profiling.
Smith said that, as a mainframe user, the concept of the root account with its unlimited power to wreck the entire system was frightening beyond belief. On the other hand, he added, the Unix notion of "everything is a file" was great -- consider how much time, he told the audience, you spend writing custom programs just to connect different components of the system. I guess the answer to that question is always "too much."
Testing and installing
After several sessions discussing Linux on System z hardware, the next logical step is to try it out in person. For that, Richard Lewis and Chuck Morse from IBM facilitated a hands-on installation workshop in one of SHARE's computer labs.
The class was three sessions long, accounting for most of the conference day. Attendees were given access to the zSeries mainframe at IBM's booth on the expo floor, on which they could create virtual machines and install Linux. The first time through, participants installed the Marist College distribution, a minimalist 2.2-series distro that was the first public release of a complete Linux kernel for the z architecture.
Morse explained that the older distribution affords the opportunity to learn the system better by forcing participants to manually perform tasks that might be handled automatically in newer, commercial distros. And those newer distros came next; after completing the Marist installation, participants started over with either Novell's or Red Hat's System z distro -- and potentially could choose to try both, if they wanted to spend the time.
Lewis said IBM has been offering this lab at each SHARE event for the last six years. In that time, not much has changed on the Linux side of the equation -- customers are running the same applications, and lab participants still include a mix of new and old customers. He did point out one change made for this year's session; due to increasing interest from customers, this year's installations were done using SCSI storage instead of traditional mainframe DASD disks.
That single distinction is of greater importance than you might think. Customer interest in SCSI storage is entirely driven by the Linux on z/VM community; IBM does not even offer SCSI disks on its other mainframe operating systems. That means that for the first time, Linux is influencing the design of real-world mainframe hardware.