August 18, 2010

Lets draw a lowcost distributed database concept

At the latest since i operate a bunch of single cheap dedicated machines which is limiting the growth of each project, i share this common dream of a (lowcost) distributed database

Especially the idea of one which come as close as possible to the imagination of one single machine with combined power of especially 100s of disks and of course also 100 of cpu cores & terabytes of RAM to use it like an amazon S3 "flatrate" account, while its single machines are maybe distributed to several locations

To name a more conrete subject of discussion:
it will already be great if you guys can just name single pieces of Software and tell why its a must to use it or not.
At a whole this will very well be able to draw a concept in mind.

Examples :

"Apache Hadoop: obviously a good choice because its proven to work even for giants as facebook and its free&opensource"
" Open Solaris because ZFS ....? "

But the most striking addtional question is:
what if iam not able to build my own perfect cluster
but instead i just rent a bunch of cheap 100mbit storage servers at one ISP and another few at another ISP at a completely differnt locations and call them all a distrubuted database cluster --- will i be able to configure something like Hadoop to still work stable under those cirumstances?

Thanks a lot for reading :woohoo:

Click Here!