Linux.com

Home News Enterprise Computing Systems Management Quantcast Open Sources Hadoop Distributed File System Alternative

Quantcast Open Sources Hadoop Distributed File System Alternative

Quantcast, an internet audience measurement and ad targeting service, processes over 20 petabytes of data per day using Apache Hadoop and its own custom file system called Quantcast File System (QFS). Today, it’s making that technology available to as open source under an Apache license. You can now find it on GitHub.

The default Hadoop file system is called Hadoop Distributed File System (HDFS). CEO Konrad Feldman says Quantcast started using Hadoop in 2006. In 2008 as Quantcast started collecting 1TB of data per day the team realized that it was going to need a file system with better throughput than HDFS.

They settled on using the open source Kosmos Distributed File System (kosmosfs), but didn’t feel that it as production ready. So they hired Sriram Rao, the lead architect of kosmosfs, to work on making it production ready. The result was QFS, which Quantcast has been using in product for about four years now, though Rao has since left the company for Microsoft.

Read more at TechCrunch
 

Comments

Subscribe to Comments Feed

Upcoming Linux Foundation Courses

  1. LFS220 Linux System Administration
    05 Jan » 08 Jan - Virtual
    Details
  2. LFD331 Developing Linux Device Drivers
    12 Jan » 16 Jan - Virtual
    Details
  3. LFS520 OpenStack Cloud Architecture and Deployment
    12 Jan » 15 Jan - Virtual
    Details

View All Upcoming Courses

Become an Individual Member
Check out the Friday Funnies

Sign Up For the Linux.com Newsletter


Who we are ?

The Linux Foundation is a non-profit consortium dedicated to the growth of Linux.

More About the foundation...

Frequent Questions

Join / Linux Training / Board