There are plenty of valid reasons not to upgrade to the latest release of your Linux distribution. You have to reapply all manner of preferences, hunt around in the new Applications menus to find where your programs have moved to and what they're called this time around, get used to new ways of doing the simplest tasks, and there's usually a new firewall subsystem to boot. But the one that has always gotten under my skin is the fact that upgrading my distro always breaks the add-on software I've installed myself.
If I'm lucky, it's just mangled. But on many occasions the installer program removes it entirely because it judges it a conflict with something new. In fact, every support forum I've looked at flat-out recommends that you do a clean install instead of attempting an upgrade. So it's guaranteed that whenever you upgrade, you'll have to reinstall everything from media players and games to commercial software and critical applications that just aren't mainstream enough for inclusion in the distro.
More and more people these days simplify package management with tools like apt and yum, and third-party package repositories like freshrpms, livna.org and Dag Wieers'. But the repositories and the distros each act like they live in separate worlds, and when upgrade time comes, it's the user who is inconvenienced.
Mainly you see this in the way the repository applications try to install virtually everything to /usr. According to the Filesystem Hierarchy Standard (FHS), /usr belongs to the system and should not be touched after initial installation. That may be a bit strict, but at the very least, binary packages should not be installing software to /usr that is canonically not part of the Linux distribution. If they'd just make that one change, I wouldn't have to re-install Mplayer after every upgrade.
Sure, it's not totally the packagers' fault -- the majority of software projects provide RPMs that install to /usr, in disregard of the FHS. You can force an RPM to install somewhere else with the --prefix switch, but only if it doesn't have any hard-coded paths, and a lot of them do. There are options to relocate individual files at install time, but that quickly gets impractical and might not work. But bad RPMs aren't the root of the issue.
This is a problem that already has a solution, and few people follow it: using /usr/local and /opt.
Following the FHS definition, any software you install yourself not from the distro really belongs in /usr/local. /usr/local is in every user's path, it contains the same bin and lib subdirectory structure that /usr does, and it won't get overwritten or mangled by a system upgrade that writes through /usr. If you write custom scripts, compile something from source, or install something self-contained (like Thunderbird and Sunbird tarballs), it finds its home in /usr/local.
There is some debate as to whether binary packages should follow the same policy. We encounter that old multiple personality disorder issue when installing a lot of RPMs -- you have to become root to do it, so in a sense you are becoming "the system" for a few minutes. But ultimately it's still you -- not the distro -- managing the application.
So what happens when you throw a third-party repository into this equation? You get /opt. The FHS defines /opt as the home for large packages from independent vendors. It used to be common for OpenOffice.org to reside in /opt, and before that, desktop environments like GNOME and KDE. That's no longer the case, though, as they're integrated enough in a modern Linux distro to belong in the main filesystem. But third-party software from repositories doesn't fit anywhere else, and they are independent vendors.
Here is what I propose:
I know, I know; it isn't that simple. I've discussed this ad nauseum and I know the criticisms. For one thing, two repositories could both provide the same package, and you end up with two copies in different directories. That's true, but it's really no worse than having both binaries try to install to /usr; the only difference is that the user will see the conflict after installation instead of before. It's broken either way.
For another, it doesn't necessarily solve the "fresh install" dilemma -- the only way that /usr/local and /opt would be preserved through a fresh install is if they are not on the main disk partition. Well, the solution to that is right there in the problem statement: put /usr/local and /opt on a separate partition by default. If that strikes you as too many partitions, well, you could always make /usr/local a symlink to a subdirectory in /opt and cut down a little. It even makes sense logically if you consider /usr/local a "vendor" according to the FHS usage of /opt.
You can't stop a pathological user from changing a partition setting and causing his own problems, but part of the distros' behavior modification needs to be acknowledging that their customers are going to need add-on software, so they must plan for it. Far from keeping its hands off of my /usr/local, Red Hat has even officially decided not to follow the FHS because it doesn't see the need. I say wake up and stick to the standard whether you enjoy it or not.
Sadly, there is no way to enforce this good behavior. Even if major projects like Mplayer and major repositories like DAG play by these new rules, others will not. So my final suggestion is that package-management programs start building in functionality to override or rescue poorly built packages. It'd be nice to see a flag in Synaptic when a package has hard-coded file paths -- and when it doesn't, to let me choose where to install it.
The point is, we live in a world where more and more users keep their machines up-to-date through live, over-the-wire package management. And no one vendor -- Linux distributions included -- provides absolutely everything. If we don't find a way for the different vendors to coexist and cooperate on one machine, then every increase in market share is going to bring with it an increasing number of frustrated and confused new users.
In closing, I have some exercises for the reader. Look up an install or upgrade guide for the distribution you are currently using. See how many packages it tells you you'll have to download separately when you're finished. Count how many of these you already have installed.
When you're done with that one, see if there is anything in /opt. On my system, the only packages that have installed themselves into /opt are Yahoo! Messenger and Quake 2. Kudos to the nice people who brought us both of those. If only everyone else would play nice, too.
Note: Comments are owned by the poster. We are not responsible for their content.
Why trust a commercial entity to maintain your Free Software?
I give courses on Linux, and I found out only one reason to do this : certifications for certain hardware and software.
For the rest, I also find that Debian is the best, although I need to run RH9 on my server, because it is the only distribution which has a kernel that supports the Promise SX6000 controller.
But that should be fixed according to Markus Lidel, who maintains the I2O subsystem.
If this works, then midrange server hardware is not exclusive for Red Hat and SUSE anymore.
From my reading of the file hierarchy I interpret<nobr> <wbr></nobr>/opt and<nobr> <wbr></nobr>/usr/local to contain the following:
<nobr> <wbr></nobr>/opt - packages (rpm, tar, etc) of third-party software.
<nobr> <wbr></nobr>/usr/local - all local applications not in the distribution, including the output of the<nobr> <wbr></nobr>/opt packages after installation utilities are run.
In this way<nobr> <wbr></nobr>/opt becomes a third-party package management locale rather than an executable location.
Finally, developers need to do better at detecting and adding/using libraries for their software. In my mind a package should contain everything needed to run an application, and the make files/package files should handle the details of compatability for the end user - loading a library only if a compatible version does not exist, and creating a new directory, rather than overwriting an existing installation. Upon seeing this it should build the application configuration file to take this into account instead of using the LIBPATH etc. environmental variables stupidly. This could be as easy as putting a 'LIB=/mylib/dir' to the file - and using that, rather than the environmental defaults if so found. 90% of my headaches integrating non-standard packages into distributions would be solved if everyone did this.
When doing a major upgrade on a server, I usually do a clean install onto a separate partition (or a different machine altogether, especially if it's the firewall); that way I always have a working system to fall back on. After that I can spend a week or two in a chroot getting it right.
Of course, these days I mostly just do
<TT>apt-get dist-upgrade</TT>but it's fun to play with new distributions every once in a while.
I've had a good laugh at your problems<nobr> <wbr></nobr>:-) After reading your list I can see that nothing has changed since RedHat 5.x days and these problems are why, after N-th clean reinstall, I decided to dump this broken distro.
You write:
There are plenty of valid reasons not to upgrade to the latest release of your Linux distribution. You have to reapply all manner of preferences, hunt around in the new Applications menus to find where your programs have moved to and what they're called this time around, get used to new ways of doing the simplest tasks, and there's usually a new firewall subsystem to boot. But the one that has always gotten under my skin is the fact that upgrading my distro always breaks the add-on software I've installed myself.
Gee, just move on to some REALLY easy to use and, even more important, upgrade distro. Like Debian for example. Since moving to Debian 5 years ago I almost forgot what does it mean to have problems with upgrading and 'RPM dependency hell' is a nightmare of the past. I've had to reinstall system once in that time because of a disk crash, and it's always clean and usable after all this time regardless of how many packages I install/uninstall or how many times I upgrade the system.
* Every application project that offers independent downloads should package their RPMs to install in<nobr> <wbr></nobr>/usr/local.
No! There are a bunch of different base paths, like<nobr> <wbr></nobr><tt>/</tt>+<tt>/usr</tt>,<nobr> <wbr></nobr><tt>/usr/local</tt>,<nobr> <wbr></nobr><tt>/opt</tt>,<nobr> <wbr></nobr><tt>/pkg</tt> and so on. These should be handled by different *mechanisms*, not different packagers. Most importantly, RPMs should never ever touch<nobr> <wbr></nobr><tt>/usr/local</tt>. That's where files ends up if you<nobr> <wbr></nobr><tt>./configure && make && make intall</tt>, and you must keep those files separate from the RPMs, or you end up with an unmaintainable mess.
* Third-party repositories should package their RPMs to install in<nobr> <wbr></nobr>/opt, under their own subdirectory.
No! It won't solve any problems. The third party packages should be well integrated in the distro, or not be installed at all. And it shouldn't change it's install location based on who did the packaging.
* Distros should stick fast to the FHS definition of<nobr> <wbr></nobr>/opt and<nobr> <wbr></nobr>/usr/local, and never touch them.
And so they do. So that you can install Matlab in<nobr> <wbr></nobr><tt>/opt</tt> and that Firefox tarball in<nobr> <wbr></nobr><tt>/usr/local</tt> without worrying about overwriting or being overwritten by RPMs.
The situation just isn't as bad as you make it out to be. Just use non-sucky repos.
The spirit of the FHS is to separate the OS from third party packages, for easier maintenance. But with RPM:s you don't need to do that. Really. You don't.
But it is still useful to separate RPM installations in<nobr> <wbr></nobr>/usr from other types of installations in<nobr> <wbr></nobr>/usr/local. And RH recommends that. All in the spirit of the FHS.
Think of it like this: RPM is the OS, in the FHS sense. If it's an RPM, it goes in<nobr> <wbr></nobr>/usr. If it's not an RPM, it's not in the OS and should go in<nobr> <wbr></nobr>/usr/local or<nobr> <wbr></nobr>/opt.
So all you RH suxxors dweebs can go home now.
'But the repositories and the distros each act like they live in separate worlds, and when upgrade time comes, it's the user who is inconvenienced.'is also mentioned by ESR in his Fedora Multimedia Installation Howto. This is another crucial issue with regard to this problem. If we enter into a world of package conflicts, nothing but bad is going to happen.
'Damned Things that Fedora Core won't carry'. The independant packagers do a fantastic job as it is, if they actually pooled their respective resources the results could be, in my view, well worth the effort. Particularly in the long term. This is a project that is supposedly under way, under the catchy title of RPMForge.
Did that today
Posted by: Anonymous Coward on December 18, 2004 02:24 AMAll my preferences, including my own theme, my spatial nautilus windows, mail accounts, bookmarks, C# and pascal projects worked like a charm.
btw. I exchange my notebook for every new distro (started this ritual with RH 6.2) and not even once there was a problem. Speed gain with FC3 is enormous.
#