Home News Enterprise Computing Systems Management Quantcast Open Sources Hadoop Distributed File System Alternative

Quantcast Open Sources Hadoop Distributed File System Alternative

Quantcast, an internet audience measurement and ad targeting service, processes over 20 petabytes of data per day using Apache Hadoop and its own custom file system called Quantcast File System (QFS). Today, it’s making that technology available to as open source under an Apache license. You can now find it on GitHub.

The default Hadoop file system is called Hadoop Distributed File System (HDFS). CEO Konrad Feldman says Quantcast started using Hadoop in 2006. In 2008 as Quantcast started collecting 1TB of data per day the team realized that it was going to need a file system with better throughput than HDFS.

They settled on using the open source Kosmos Distributed File System (kosmosfs), but didn’t feel that it as production ready. So they hired Sriram Rao, the lead architect of kosmosfs, to work on making it production ready. The result was QFS, which Quantcast has been using in product for about four years now, though Rao has since left the company for Microsoft.

Read more at TechCrunch


Subscribe to Comments Feed

Upcoming Linux Foundation Courses

  1. LFS201 Essentials of System Administration
    12 Jan » 30 Mar - Online Self-Paced
  2. LFD320 Linux Kernel Internals and Debugging
    13 Jul » 17 Jul - North Reading - MA + Virtual (GUARANTEED TO RUN)
  3. LFS426 Linux Performance Tuning
    13 Jul » 16 Jul - Virtual (Guaranteed to Run)

View All Upcoming Courses

Become an Individual Member
Check out the Friday Funnies

Sign Up For the Newsletter

Who we are ?

The Linux Foundation is a non-profit consortium dedicated to the growth of Linux.

More About the foundation...

Frequent Questions

Join / Linux Training / Board