Linux.com

Home Learn Linux Linux Documentation The Not-Ready Btrfs and ExFAT Linux Filesystems

The Not-Ready Btrfs and ExFAT Linux Filesystems

Two newer filesystems of importance to Linux are exFAT and Btrfs. exFAT is the controversial Microsoft filesystem for Flash memory devices, and Btrfs is for "big data". Once upon a time there was much sound and fury around these, but lately it's been quiet, so let's see what's been happening.

exFAT

In this glorious year 2012 in the new millennium, it is rather astonishing that there is not a common, open, unencumbered filesystem for Flash storage devices. Cameras, game consoles, music players, phones, and USB storage devices are everywhere, and it makes no sense to make something as fundamental as a filesystem into a roadblock. But it is, thanks to our good friends in Redmond.

536px-Betacam SP Camcorder 01 KMJSony Betacam Camcorder. Image courtesy Wikimedia Commons.

Microsoft's creaky old FAT filesystems, FAT16 and FAT32, have long been the de facto standard filesystems for Flash storage devices. They enable portability because FAT is supported on all major operating systems, and they don't have access controls so there are no permissions hassles-- just plug in your device and use it. But despite FAT's age and ubiquity, Microsoft successfully enforced its FAT patents against TomTom in 2009. TomTom agreed to drop FAT32 support from their products, several of which were built on Linux. Microsoft has also gone after Android vendors, such as Motorola, who use FAT.

The legal landscape, as always, is bizarre. Linux can support FAT32 without paying royalties because of an inane technicality: long and short filenames. My fellow old codgers recall the 8.3 DOS filename convention: filenames could be no more than 8 characters long with a 3-character extension. This collided with grownup filesystems that supported longer filenames, which FAT truncated. And that is why something like nicelongfilename.txt would be shortened to nicelo~1.txt.

When Microsoft finally figured out how to support longer filenames (up to 255 characters), they were able to patent this long filename extension to FAT. When Microsoft sued TomTom it caused a fair bit of worry that Linux developers and vendors would become litigation targets. Andrew Tridgell (one of the lead Samba developers) wrote a kernel patch to disable the creation of 8.3 short filenames. This was enough to dodge the long-filename patent, which covers implementations that support both short- and long-filenames. (I am not making this up; see for yourself.) If you need 8.3 support, use the msdos filesystem driver.

FAT32's usefulness has been declining for some time anyway, because it cannot support files larger than 4GB, it's inefficient, and it becomes more inefficient with larger volume sizes. And so around 2009, there was a golden moment when manufacturers of all the nifty devices we use could have banded together and developed an open common filesystem. There are multiple benefits to this approach: data portability, ease of use, community support, and no royalties. It could even have been based on any of the excellent existing open source filesystems.

But no. Microsoft blew the dust off exFAT, which was originally developed for Windows Embedded CE and is essentially FAT64, and made deals with the SD Association and hardware vendors. So SDXC memory cards are formatted with exFAT because it is part of the SDXC specification, and this month Sharp, Sigma, NextoDi, Black Magic and Atomos Global signed agreements to use exFAT on their products. This includes high-end camcorders, Android tablets, and digital cameras.

Linux users have options, sort of. Tuxera sells a good exFAT driver, but only to OEMs, such as Android vendors. There is a free exfat driver, fuse-exfat, and it is included in several distros. This is built on fuse, filesystem in userspace. I've tested it a bit without problems, but the developers do not have access to any specifications and it's still young, so it has some rough edges. I would not rely on it for syncing a Linux PC with devices that use exFAT, like cameras and smartphones.

Butter Filesystem

The Butter/Better/B-tree Filesystem, Btrfs, has elevated the hopes of filesystem nerds and kept us in a state of suspense, because it was supposed to be production-ready earlier this year. As 2012 fades away it isn't ready-- it is still considered experimental by the Linux kernel developers, even if it is the default filesystem in Oracle Unbreakable Linux. Perhaps the Oracle Unbreakable Enterprise Kernel has extra-rugged Btrfs-fu.

Rvk mun arch file cabinets 2011Massive file cabinets. Image courtesy Wikimedia Commons.

Me, I take my filesystems seriously, because a buggy filesystem eats data. (Like the recent Ext4 bug, oops.) I wonder if Oracle is as fast to fix bugs as the Linux kernel team? At any rate there are a few ways to try it out. The easiest way is to run a Linux distribution that includes it, like Debian, Ubuntu, Gentoo, OpenSUSE, Fedora, and Arch. Btrfs is already built into the Linux kernel, so all you do is install the userland tools.

Distros are always going to lag behind the latest builds, so you can snag the latest versions from the git repo. If you go this route you'll also need the latest kernel.

The best introduction to administering Btrfs is a two-part series on Oracle's Technet: How I Got Started with the Btrfs File System for Oracle Linux and How I Use the Advanced Capabilities of Btrfs. Filesystems are fascinating, and Btrfs promises to be the big-time datacenter Linux filesystem with advanced built-in management tools and über-reliability.

 

Comments

Subscribe to Comments Feed
  • Rene Bon Ciric Said:

    I'd like to say I am a Btrfs user on production. It is awesome and, if it weren't for that Anaconda bug on Fedora 17, I'd be using it as my default filesystem. It is awesome. I would recommend: http://www.funtoo.org/wiki/BTRFS_Fun It's the best show-off tutorial I've found ;)

  • K. Darien Freeheart Said:

    "it is still considered experimental by the Linux kernel developers, even if it is the default filesystem in Oracle Unbreakable Linux. Perhaps the Oracle Unbreakable Enterprise Kernel has extra-rugged Btrfs-fu." Not anymore, but when that decision (and the design for that product was drafted) they did - Chris Mason. "Me, I take my filesystems seriously, because a buggy filesystem eats data. (Like the recent Ext4 bug, oops.)" Interestingly, I've never had a data loss caused by anything other than human negligence, and I've been running BTRFS for a long time. There was an early bug where files would accidentlaly get replaced with empty files. After that bug was closed, I began using btrfs and have no had issues. Ext4 has. Which is odd, but also why it pays to not be on the "cutting edge" if you take data integrity seriously.

  • Daniel Cordey Said:

    I've been using btrfs without any problem for more than a year. However, It's totally useless as a root file system with any "apt" update mechanism (Debian based). The reason ? 'apt' execute fsync() like crazy and btrfs doesn't like it... at all (claiming non-POSIX compliance) It's one of these stupid "mine is bigger" fight which forced me to de-install btrfs from all our systems (at work). I'm thrilled to use btrfs and all its feateures, but will wait until one of (or both) the btrfs/apt team decides to stop this "baby" fight. It bothers end-users and hurts GNU/Linux/FOSS image.

  • stib Said:

    The betacam camera you have pictured there is actually an analogue tape camera. SP Betacam was the industry standard for television production before DV, and many edit suites (like mine) are still filled with rows of grey SP tapes. But there won't be anywhere to put an SD card into it - apart from anything else, SP Betacam pre-dates FAT32 by about ten years.

  • Steve C Said:

    I used BTRFS for a while, because it is efficient - on a backup file server. The issue is that RSYNC would get so slow that it was literally crawling for hours. EXT4 is done in minutes. Now after about 18 months with EXT4, I'm curious to see if those issues have been fixed but it's too much time to invest.

  • Matthias Eckermann Said:

    The larger Linux distros not only include it, but many (like SUSE) actively support and contribute to refine and harden btrfs. When you integrate btrfs with a tool like Snapper, for instance, it can compare snapshots and revert differences between snapshots. This allows root and non-root users to view older versions of files and revert changes. Snapper is available as a command line tool and a YaST module. For more on Snapper, check out http://snapper.io/ -Matthias, SUSE

  • Ron Said:

    For portable storage, FAT32 is probably the best. Even though we Linux people don't like it, it works on every operating system. Try to get M$ to support an open filesystem format. Good luck with that. They write their own TCP stack, their own browser, their own everything -- even though there's plenty of open source code available under BSD, MIT and Apache licenses which would allow them to get their job done faster, better and more cheaply without sacrificing one bit of their IP rights. M$ still thinks they're the market leader. Of course they're not. We Linux people, we've won. Android has completely taken over, and there's little chance of a tide change in the near future. People love Android, and they very well should -- it's great! But as for these chintzy little USB flash drives and SD cards, don't fret over them using some junky little MS filesystem. They're not designed for high performance or high security. It's a way to get data from one place to the next. If you want to maintain Linux permissions, make a tarball and put that on the USB stick. You'll be fine. Slower than Christmas, but still fine.

  • Erinn Volkner Said:

    FAT32 does not support files larger than 4GB, so it's no good for movies and large audio projects. If your shiny new exFAT camera or smartphone doesn't come with the Tuxera driver, then it's unusable with a Linux PC. exFAT is just another parasitical Microsoft tollgate. TomTom caved in and Android vendors all pay protection racket fees to MS. yah winning uh huh.

  • Erinn Volkner Said:

    We've won, eh? Android is only sorta-open, and most users have no clue it's Linux. It's just another sortaLinux trapped on closed hardware, like Tivo. Do you see Linux PCs on the shelves of Best Buy or Fry's? Has Dell finally figured out how to release and sell a Linux PC without apologizing, or plastering "We recommond Windows" all over their pages? Hey, how about that Windows UEFI Secure Boot that locks out Linux? And oh yeah, exFAT with no meaningful Linux driver. We haven't won anything, but merely developed great skill in workarounds. That is not winning.

  • MH Said:

    UDF is open, portable and supports big drives and files.

  • Leslie Satenstein Said:

    Fedora 18 and BTFRS. For home system, all is working just fine. Yesterday, the latest Kernel RC candidate included some btfrs improvements that increased throughput by up to 50%. For SSD devices, even more. Btfrs is here to stay. I don't find a need to return to Ext4.

  • Sofia Koutsouveli Said:

    The main problem with FAT is its inability to manage files larger than 4GB. exFAT solves that issue, but it's a controlled technology by Microsoft. Perhaps GNU and Linux developers should focus on creating their own version of FAT64 and promote it as a free alternative.

Upcoming Linux Foundation Courses

  1. LFD312 Developing Applications For Linux
    05 Jan » 09 Jan - Virtual
    Details
  2. LFS220 Linux System Administration
    05 Jan » 08 Jan - Virtual
    Details
  3. LFD331 Developing Linux Device Drivers
    12 Jan » 16 Jan - Virtual
    Details

View All Upcoming Courses

Become an Individual Member
Check out the Friday Funnies

Sign Up For the Linux.com Newsletter


Who we are ?

The Linux Foundation is a non-profit consortium dedicated to the growth of Linux.

More About the foundation...

Frequent Questions

Join / Linux Training / Board