February 26, 2007

Krugle offers code search engine for open source, with open source

Author: Tina Gasperson

With the rise in popularity of open source software, developers don't need to start from scratch when coding new software. Instead, they can use specialized search engines that crawl repositories to find the perfect code snippet. Now, one entrepreneurial open source developer has built a business that expands on the basic code search engine, and in true hacker recursive style, finds his company relying on the very tool it exists to create.

Krugle is a combination code search engine and developer community. It crawls some 80,000 open source projects in dozens of different programming languages, and even allows searchers to look at code snippets from any of Safari's online programming books. One of the things that Ken Krugler, founder and CTO of Krugle, says makes the search engine unique is the built-in ability to collaborate. "You can put notes on the code, assemble sets of Web pages and code files, and put them into a 'code space.'" Developers can share their information collections with other members of Krugle, further reducing the need to "recreate the wheel."

Krugle itself is almost completely open source. Its 120 servers run on Fedora Core, and an array of open source tools helps Krugler and his staff extend and maintain the Krugle crawler. It's based on Nutch, the same engine that Yahoo! began using (and pouring a lot of money into) early in 2006. Krugle also uses Hadoop, an open source framework that is ideal for running search engine applications on commodity clusters. Krugler says that, while there's been no need for his company to contribute code back to Nutch, they did fund a developer to add some custom event tracking capabilities to Hadoop.

Krugler starting thinking about a source code search engine in 2005, after he'd been exposed to OSS while helping out on the Chandler open source PIM project as an internationalization specialist. "I got involved because I wanted that software," Krugler says. "That's where I became aware of two things. First, just how much open source software is available, and how quickly it is being developed. Second, just how hard it was to find the things I was looking for.

"I decided a search engine for developers was something really interesting and useful. I was having a conversation with a friend, and he told me the most powerful tool his developers had was the ability to search for code. I started thinking about that."

Krugler believes that code search is such a powerful tool because "it gets you the information that has incredibly high leverage. If by searching, you don't have to write the code or debug it, that saves time and money. That's fascinating." He did an informal study of developers in his network and found that the average number of code searches they performed every month was around 200.

Because the business is built on open source, Krugler was able to bootstrap it for "a couple thousand dollars." Since then, investors have provided two rounds of funding, but not before Krugler convinced them that "intellectual property" and "defensible positions" weren't crucial to creating a potentially profitable company. "Open source is the base," he says. "You can move up the stack, write proprietary add-ons to things."

Krugle offers a free search engine and forums, selling advertising to make money, but it also provides a commercial version of its search technology. "It gives you the same kind of functionality, just inside the company's firewall." Krugler says customers can specify restrictions on searches, such as excluding code that is released under certain licenses that may conflict with the company's existing code base.

Krugle says his developers actually use Krugle internally to help them enhance the product itself. "That's one of the fun things about working on it."

Other entrepreneurs who might want to build a company on open source should know what they're getting into, Krugle says. "It does typically assume a level of expertise. A lot of times there may not be the same level of support as with commercial. If you're not technical, you need to find somebody to work with who is, or make sure that support is available through consultation. I see a lot of people posting questions saying, 'I'm not really that tech knowledgeable.' That can be tough.

"Having said that, for a lot of entrepreneurs, using open source could mean the difference between success and failure. It does let you significantly reduce your cash requirement, and in some cases it may be the only way for you to do what you want to do. If you need specialized software, getting something from a vendor may cost too much or take too long."

Click Here!