Author: Mayank Sharma
Metalink is an open standard that claims to make downloading easier, faster, and more reliable by helping users extract the last drop of juice out of their connection. But Metalink isn’t your run-of-the-mill download accelerator. It is in fact a framework for use by other download clients, and bundles traditional HTTP and FTP methods of downloading files along with BitTorrent.
The Metalink standard replaces static URLs with a .metalink file, which is a simple XML file. The file in turn contains locations of all the mirrors of the application you want to download. In addition to HTTP and FTP mirrors and rsync, Metalink supports several P2P methods as well, including BitTorrent, ed2k, and magnet links. For example, the OpenOffice.org metalink contains links to more than 50 HTTP and FTP servers and a torrent.
If I were using a regular hyperlink to download OpenOffice.org and the server went down midway through the transfer, I would be left with an incomplete transfer that could either be resumed or not depending on the download client. But with Metalink, if one server goes down, the client software simply jumps to another mirror and resumes downloading from that point. In effect, this increases reliability, as all listed servers would need to be down for the file to be unavailable.
Files downloaded with metalinks are automatically verified. Metalink supports both MD5SUM and SHA1SUM checksums as well as PGP signatures, which are embedded in the .metalink file itself. Checksums are unique for every file, like a fingerprint. If there’s an error in transfer, or if someone has maliciously replaced a good file with a bad one, the checksums won’t match. If that happened with another file transfer method, you’d have to download the file again from another mirror. With Metalink, if the application you are downloading has a torrent, Metalink can use a torrent’s partial file or chunk checksums to verify mirror downloads as well. If only a small chunk of a download has errors, Metalink just re-downloads that part instead of the whole file.
Generating metalinks
All software publishers need to do to let users download files via Metalink is to generate a .metalink XML file with a list of download locations and assign each a priority. While the format is simple enough to be hand-coded, there is an online Metalink Creator that does the job for you. Just fill in the required information and presto, your very own metalink.
Apart from the online tool, Metalink provides several cross-platform applications and Perl scripts that can be used as well.
Once a file is generated, publishers can add a MIME type to their Web server (“application/metalink+xml”) which will help users open the metalink with their download client. If it isn’t set, the .metalink file will be displayed as text in a client’s browser upon clicking the Metalink URL, but Metalink clients will still work if you point them to the URL for the metalink.
Software publishers can prioritize individual download locations. By giving torrents a higher (or maximum) priority, publishers can save bandwidth, but still ensure delivery even if all seeds stop sharing, by falling back on FTP/HTTP.
Using Metalink
Downloading with Metalink isn’t any different from downloading through a regular download client. You need to get a download manager that supports Metalink. Currently, Linux users will have to either use the command-line aria2 client, the graphical wXDownload Fast client, or the Flashgot Firefox plugin. KGet2 has metalink support in its to-do list for KDE4. There are clients for other platforms as well.
Once you have the download client, check whether the project you are about to download uses metalink. OpenOffice.org has been supporting Metalink since July. packages.ro provides Metalinks for several major distributions and applications.
In addition to downloading ISO and individual software packages, Metalink can be used for updating packages as well. Arch Linux is planning to use Metalink in its pacman package management application to ensure quick and reliable delivery of updates.
Re-implementing features
Though Metalink remains relatively unknown, Anthony Bryan created the utility 11 years ago. He had to trim several features to get download managers to implement it, because they thought some features were going to be more work to program and would not be useful for most people. One feature he took out was the ability to select a mirror based on its load. When it works, the priority setting can be complemented by generating metalinks depending on the server’s load. There’s also an update URL in the Metalink header, so clients can check to see if mirrors have been removed or added.
Another absent feature is the ability to generate a Metalink that lists several versions of the file you want to download along with information on their language and operating system. While downloading, the correct version could be filtered by saved settings or command-line switches. For example, the Firefox download page lists downloads for three platforms in several dozen languages. All these links could be replaced by one Metalink that selects the appropriate version to download based on a client’s language setting and operating system.
Bryan hopes that Metalink becomes popular enough for him to re-implement these features, but for that he needs support from download clients and software publishers.
Conclusion
I was pretty impressed by Metalink. It saves users from wasting time selecting a fast mirror and ensuring the integrity of a download. By grabbing chunks of files from multiple sources at once, it makes efficient use of available bandwidth. Plus, it’s newbie-friendly, as there isn’t any user interaction involved. I hope to see more distributions, software publishers, and download clients offer Metalink downloads.
