Linux.com

Recently, Byrne began switching Framestore's Network File System (NFS) servers out for machines running the Lustre open source cluster file system. "We have quite a lot of machines, and our render farm creates a lot of I/O load on servers when it comes to pulling in large 3D objects, doing a lot of computation, and spitting out frames at the other end," he says. "When you have 500 machines doing that to say, one NFS server, it really doesn't work. NFS wasn't designed for clusters."

Framestore needed to be able to increase the size of its render farm without having to worry about current and future bandwidth needs. And it didn't want to have to buy specialized hardware, especially since it had already heavily invested in big Hewlett-Packard file servers. "We have a lot of these lying around, and we wanted to redeploy them in a more efficient way." Byrne looked at other high-end storage servers, such as those from Bluearc -- "It was just like a fast NFS server and wasn't scalable" -- and Network Appliance -- "It's three times an NFS server."

Framestore picked up on Lustre in 2002, in its infancy, Byrne says. "It wasn't really a proven commercial product. It required a lot of testing and playing around, and it took a long time to convince people in the company to have faith. It took about a year of looking at it before we were confident enough to let production start." Byrne had planned ahead for that day by configuring everything and installing what he needed to on everyone's machines.

Byrne has 16 of 60 servers converted to Lustre so far, and as current projects finish and free up storage, the staff pulls in more servers to the new system. There have been some challenges along the way, mostly from patching the Red Hat kernels with Lustre updates, and discovering and fixing bugs. "Because we started with Lustre from an early stage there was not as much documentation help," Byrne says, "nor as many people working on it as there are now. A lot of the figuring out was done by us and other [users]. That's changed quite a bit now."

Byrne says the major benefit for Framestore in open source software has been the ability to work on complicated 3D shots that require an "awful lot of I/O" without spending too much money. "It provides more I/O than we need, and that's without using all the servers we have now."

Another benefit is the cost savings, which allows Framestore to compete with big studios like Pixar and Dreamworks. "We're not in the same league as some of them," Byrne says. "If they want to build the newest and best infrastructure, they can click their fingers and the money will appear. We're not that well financed, so we have to do things a lot cheaper and a lot faster in order to stay competitive. I think open source gives us that ability -- that's been the great thing."

Share    Print    Comments