December 17, 2004

Why I won't upgrade my Linux distribution

Author: Nathan Willis

I'm not upgrading my Fedora Core 2 machine to Core 3, even though the new version has been out for a couple of months. There's not anything wrong with FC3 itself, it's just that system upgrades are both a blessing and a curse. I guess that's one of those dirty little secrets every Linux user knows, but that none of us talks about.

There are plenty of valid reasons not to upgrade to the latest release of your Linux distribution. You have to reapply all manner of preferences, hunt around in the new Applications menus to find where your programs have moved to and what they're called this time around, get used to new ways of doing the simplest tasks, and there's usually a new firewall subsystem to boot. But the one that has always gotten under my skin is the fact that upgrading my distro always breaks the add-on software I've installed myself.

If I'm lucky, it's just mangled. But on many occasions the installer program removes it entirely because it judges it a conflict with something new. In fact, every support forum I've looked at flat-out recommends that you do a clean install instead of attempting an upgrade. So it's guaranteed that whenever you upgrade, you'll have to reinstall everything from media players and games to commercial software and critical applications that just aren't mainstream enough for inclusion in the distro.

More and more people these days simplify package management with tools like apt and yum, and third-party package repositories like freshrpms, and Dag Wieers'. But the repositories and the distros each act like they live in separate worlds, and when upgrade time comes, it's the user who is inconvenienced.

Mainly you see this in the way the repository applications try to install virtually everything to /usr. According to the Filesystem Hierarchy Standard (FHS), /usr belongs to the system and should not be touched after initial installation. That may be a bit strict, but at the very least, binary packages should not be installing software to /usr that is canonically not part of the Linux distribution. If they'd just make that one change, I wouldn't have to re-install Mplayer after every upgrade.

Sure, it's not totally the packagers' fault -- the majority of software projects provide RPMs that install to /usr, in disregard of the FHS. You can force an RPM to install somewhere else with the --prefix switch, but only if it doesn't have any hard-coded paths, and a lot of them do. There are options to relocate individual files at install time, but that quickly gets impractical and might not work. But bad RPMs aren't the root of the issue.

This is a problem that already has a solution, and few people follow it: using /usr/local and /opt.

Following the FHS definition, any software you install yourself not from the distro really belongs in /usr/local. /usr/local is in every user's path, it contains the same bin and lib subdirectory structure that /usr does, and it won't get overwritten or mangled by a system upgrade that writes through /usr. If you write custom scripts, compile something from source, or install something self-contained (like Thunderbird and Sunbird tarballs), it finds its home in /usr/local.

There is some debate as to whether binary packages should follow the same policy. We encounter that old multiple personality disorder issue when installing a lot of RPMs -- you have to become root to do it, so in a sense you are becoming "the system" for a few minutes. But ultimately it's still you -- not the distro -- managing the application.

So what happens when you throw a third-party repository into this equation? You get /opt. The FHS defines /opt as the home for large packages from independent vendors. It used to be common for to reside in /opt, and before that, desktop environments like GNOME and KDE. That's no longer the case, though, as they're integrated enough in a modern Linux distro to belong in the main filesystem. But third-party software from repositories doesn't fit anywhere else, and they are independent vendors.

Here is what I propose:

  • Every application project that offers independent downloads should package their RPMs to install in /usr/local.
  • Third-party repositories should package their RPMs to install in /opt, under their own subdirectory.
  • Distros should stick fast to the FHS definition of /opt and /usr/local, and never touch them.

I know, I know; it isn't that simple. I've discussed this ad nauseum and I know the criticisms. For one thing, two repositories could both provide the same package, and you end up with two copies in different directories. That's true, but it's really no worse than having both binaries try to install to /usr; the only difference is that the user will see the conflict after installation instead of before. It's broken either way.

For another, it doesn't necessarily solve the "fresh install" dilemma -- the only way that /usr/local and /opt would be preserved through a fresh install is if they are not on the main disk partition. Well, the solution to that is right there in the problem statement: put /usr/local and /opt on a separate partition by default. If that strikes you as too many partitions, well, you could always make /usr/local a symlink to a subdirectory in /opt and cut down a little. It even makes sense logically if you consider /usr/local a "vendor" according to the FHS usage of /opt.

You can't stop a pathological user from changing a partition setting and causing his own problems, but part of the distros' behavior modification needs to be acknowledging that their customers are going to need add-on software, so they must plan for it. Far from keeping its hands off of my /usr/local, Red Hat has even officially decidednot to follow the FHS because it doesn't see the need. I say wake up and stick to the standard whether you enjoy it or not.

Sadly, there is no way to enforce this good behavior. Even if major projects like Mplayer and major repositories like DAG play by these new rules, others will not. So my final suggestion is that package-management programs start building in functionality to override or rescue poorly built packages. It'd be nice to see a flag in Synaptic when a package has hard-coded file paths -- and when it doesn't, to let me choose where to install it.

The point is, we live in a world where more and more users keep their machines up-to-date through live, over-the-wire package management. And no one vendor -- Linux distributions included -- provides absolutely everything. If we don't find a way for the different vendors to coexist and cooperate on one machine, then every increase in market share is going to bring with it an increasing number of frustrated and confused new users.

In closing, I have some exercises for the reader. Look up an install or upgrade guide for the distribution you are currently using. See how many packages it tells you you'll have to download separately when you're finished. Count how many of these you already have installed.

When you're done with that one, see if there is anything in /opt. On my system, the only packages that have installed themselves into /opt are Yahoo! Messenger and Quake 2. Kudos to the nice people who brought us both of those. If only everyone else would play nice, too.


  • Linux
Click Here!