Linux.com

Feature

Database vendors are joining the open source party

By Jay Lyman on December 20, 2004 (8:00:00 AM)

Share    Print    Comments   

Whether companies sell or provide their databases with fully available source code, through a new open source project, or with their very own open source license, there is definitely open source fever in the database business these days. Older, established database makers such as IBM, Sybase, and CA are releasing the code for what were previously proprietary products. But these players are coming late to the party thrown by MySQL, PostgreSQL, BerkelyDB maker Sleepycat, and Firebird. How much open source beer are these newcomers bringing to the database bash, or are they simply coming in and asking where the cups are?

The companies that create and sell databases differ on how much of the trend toward opening proprietary products under open source licenses is business and technical versus how much is marketing lip service to the open source bandwagon, which is sometimes viewed as a new opportunity for old products.

Oracle: Open source hucksters and challenge

Oracle Vice President of Technology Marketing Robert Shimp, whose company is among the only database providers not trending toward open source in some way, was critical of some open source moves by database makers in an interview with NewsForge. Shimp did not name names, but he noted that while Oracle welcomes the market growth and competition from open source databases, much of the open source database noise is centered on "orphanware."

Shimp cites companies "using open source because they see it as a marketing mechanism -- a tool for creating hype or awareness of older products. This is 'orphanware' -- software they want to abandon that has no real commercial value, so they put it out and see what happens."

Shimp elaborated by dividing open source database strategies into two categories: "serious" open source databases that provide transparency, allowing developers and users to learn and share; and the "hucksters" putting out abandonware.

In terms of competition from open source, Shimp said Oracle views the other databases as an asset in bringing new database users to the market, calling Oracle's biggest competitor the filing cabinet. Quite often, users are introduced to databases through a free or open source database, then move to Oracle as their needs become greater, according to Shimp, who called innovation Oracle's challenge and advantage.

"I'm confident we'll be able to create cool things that will get people to use Oracle," he said. "But I love the challenge the open source guys are providing."

IBM: Seeding the market

IBM program manager Les King, who touted Big Blue's move to open its Cloudscape database through the Apache incubator project Derby, said he took exception to the idea that Cloudscape was a case of abandonware. "It had a very thriving life on its own before we decided to open source it," King said.

He also indicated that like Oracle, IBM sees open source as a way to gain more market share by catering to developers with open source databases.

"There is an opportunity for vendors offering base code to hopefully seed their own market," King said. "Certainly, if we consider the seeding play, you need something to seed and Derby is perfectly set up to start seeding DB2 Express."

While there is talk of IBM open sourcing parts of its full-featured, enterprise-class DB2, King discounted the idea, referring to the complexity and value of the code.

"If you take the multiple millions of lines of code [in DB2], it naturally doesn't lend itself to dumping all of that code out there," King said. "In addition, today, there is a lot of intellectual property in the software we sell, and we wouldn't want to make it all open source."

King did anticipate more open source moves from more database players, however.

"I think you'll start to see more choice as companies do take pieces of software and make it open source because that's what they're targeting," King said, referring to developers. "It doesn't seem to be slowing."

IBM and Oracle say they are happy to see open source bring new users to the overall database market, but OS database players say that their users prefer to stick with open source, which is now rivaling DB2 and Oracle, even at the higher level.

CA: Commodity competition

Computer Associates Senior Vice President of Development Tony Gaughan referred to a sort of seeding, indicating his company -- which this year released its Ingres database under its own Trusted Open Source License, sees open source as a chance to increase mindshare and foster innovation by collaborating with its community.

Gaughan, who said the Ingres open sourcing is a return to the database's "roots" as an open source project at UC Berkeley, also pointed to the database as another instance of commoditization.

"Customers do not set out to buy a database, they purchase an application that requires a database," Gaughan said.

"We have seen a need and demand for an enterprise-class, open source database solution," Gaughan said. "MySQL is suited to read-only operations and serving up HTML content; PostgreSQL has a much richer feature set but has scalability problems and doesn't have a company behind it providing enterprise-level support; Ingres has a mature, proven, scalable transactional database, and includes clustering, peer-to-peer replication, and distributed query support."

In response to calls that the CA Trusted Open Source License is not OSI-approved and the Ingres moves are half-hearted, Gaughan said the elements not included in the available source were B1 security and the spatial object library for the database, which CA does not own.

"The security component that was removed was B1 security, which is a level of security used exclusively in situations of national security and is only available on B1 secure operating systems such as Sun CMW," Gaughan said. He added CA is working on a new 3-D, OpenGIS-compliant spatial object solution the company plans to develop with its community.

PostgreSQL: Others are too little too late

PostgreSQL core team member Josh Berkus said the open source moves by other companies are both a marketing play and done for technical reasons.

"I think that recent events in databases have vindicated the idea that open source will continue to spread through the software world, annexing one sector at a time," Berkus said in an email. "It's not a question of if software companies will need an open source strategy, but when."

While it might have been a competitive concern if Sybase and CA were making Linux and open source moves three years ago, Berkus indicated it is now the larger, older companies that are at risk from open source progress.

"My feeling is that both of these are good examples of proprietary software companies worried about being left behind by open source," Berkus said.

Berkus said databases are a likely part of the software stack to go open source because they are infrastructure software, making it easier to attract developers. However, Berkus questioned whether Sybase or CA could reap the same benefits as would a database that was born open source or free.

"In the case of CA, I'm going to reserve judgment until its license is approved by the Open Source Institute," Berkus said. "Right now, it looks like more PR than substance; unlike IBM, CA has not made Ingres separate from CA product management, which means that they're rather unlikely to attract developers. Compare the failure of both Borland's Interbase and SAP's SAP-DB as open source projects -- no offense to Firebird DB, which became a dynamic project after they forked it away from Borland. CA seems to be repeating the same mistakes."

As for Sybase, Berkus said the likely reason the competitor released the free version for Linux was a Software Development Magazine survey, which suggested PostgreSQL was pushing Sybase out of the market. Sybase did not respond to that contention.

Berkus said regardless of what other vendors are doing, the open source databases are catching up and in some instances surpassing the proprietary competition.

"PostgreSQL and even MySQL have surpassed Sybase in several areas, even if we lag behind in others," Berkus said. "PostgreSQL is particularly a threat to Sybase because our very robust, fully ACID transaction support, high availability, and support for custom statistical functions and complex queries make PostgreSQL perfectly suitable for a variety of financial applications."

MySQL: Everyone wants to be a toy

For MySQL CEO Marten Mickos, the free and open source database cavalcade from the old veterans is welcome news, and a validation that open source databases are now competing at the highest levels.

"We think it is good news for users, and we welcome these products to the open source world, Ingres, and the Linux world, Sybase," Mickos said. "We have predicted for some time that this would happen. It validates the MySQL business model. Two years ago, people said MySQL was a toy. Now, apparently everyone wants to be a toy!"

Mickos said the open source trend among databases is because of a combination of things, and is also, "a typical reaction from a large company that would like an older product to become more popular."

"Some years ago, Borland did the same thing with Interbase, and later they withdrew from the open source world," Mickos said.

However, the MySQL chief did refer to the MaxDB database, formerly known as SAP DB, as an example of older, closed code that was successfully nurtured with open source.

"MaxDB may be the only DBMS that started as closed source and was later successfully open sourced," Mickos said. "SAP AG open sourced it some years ago, and today, our company is the open source and commercial channel for MaxDB. It is a very robust, enterprise-level database and it powers an increasing number of SAP R/3 applications all over the world. It is also being used more and more by cost-conscious enterprises, by government agencies, and in developing economies. So here you have a great example that a DBMS with a long history can indeed enjoy new growth."

Sybase: Sidestepping corporate approval

While it has not released its Adaptive Server Enterprise (ASE) database as open source, Sybase has scored success with the free Linux version of its database. Citing the same database stepping stone theory as others, Sybase Senior Group Marketing Manager Amit Satoor said companies are struggling with the switch from free and open source databases to enterprise-class databases.

"Most of the customers just want low-cost access to software so they can start projects that they can deploy on a scalable platform," Satoor said.

Sybase sought to strengthen its Linux database recently with the announcement that its ASE database would run on IBM's eServer OpenPower server, a prominent Linux deployment based on the Power 5 processor.

Sleepycat: Feeling pressure from open source

Berkeley DB open source database maker Sleepycat Software's Vice President of Marketing Rex Wang said moves by Sybase, CA, and IBM were the older players' reactions to inroads from the open source newcomers.

"There's no doubt that these proprietary database vendors are being pressured to do this by the success of open source vendors," Wang said, referring to a Sybase statement that the free Linux ASE was intended to compete directly with the open source databases, as well as DB2 and Oracle. "The fact that large, incumbent, proprietary players have been motivated to make these moves indicates that the momentum is real."

Wang said Sybase, for example, felt tremendous price pressure and therefore made a restricted version of its product free to the most price-sensitive segment of its market. "Their hope is to get people to try it for free, then sell them the unrestricted version as they scale their use," he said.

But not so fast, Wang indicated, as Berkeley DB's developer focus and relative maturity -- in the market for eight years -- mean it is already appropriate for mission-critical use.

Open source getting good enough

Yankee Group senior analyst Dana Gardner said the use of a free Linux or open source database to introduce customers to a wider range of products that scale up to enterprise is a legitimate strategy. However, Gardner also said the databases that have been open source since the start may benefit from an evolving, total open source solution.

"What will be interesting is if the full stack of open source components becomes some kind of de facto standard," Gardner said. "In a best-of-breed open source approach, what are the databases that are part of that de facto standard?"

Gardner added that while he does not see open source databases such as MySQL and PostgreSQL dethroning the dominant Oracle and DB2 databases, the capabilities of the open source databases are quickly catching up and are also sufficient for many higher-level users.

"MySQL and PostreSQL -- those are quite full-featured," Gardner said. "If they continue that trajectory, good enough is good enough for many people."

Share    Print    Comments   

Comments

on Database vendors are joining the open source party

Note: Comments are owned by the poster. We are not responsible for their content.

"it looks like more PR than substance" so far

Posted by: suso_banderas on December 20, 2004 10:51 PM
Berkus is correct, I think. When I see all these new adds on the OSS websites touting the words open source, it feels more like a fluff ad directed at IT managers and business folk. Fortunately there are some businesses who backup their OSS support claims.

IBM had the substance from the start of their OSS campaign, I've been very proud of what they've done. Not so white shirt n' tie after all.<nobr> <wbr></nobr>;-)

#

you have to wonder

Posted by: Anonymous Coward on December 21, 2004 12:40 AM
When vendors release source code to older products which have minimal market share, and the codebase is large (say 1M+ LOC), does that do anyone any favors? It might be faster to re-implement from scratch than to get a development team trained on all that code - and you could architect for today's requirements, not yesterday's. Even the product's existing customers might feel they've just been told they're on their own.

#

Re:you have to wonder

Posted by: Anonymous Coward on December 23, 2004 02:27 AM
> When vendors release source code to older products which have minimal market share, and the codebase is > large (say 1M+ LOC), does that do anyone any favors? It might be faster to re-implement from
> scratch than to get a development team trained on all that code - and you could architect for today's
> requirements, not yesterday's.

Sure sometimes. But in this case (db2 probably), it has a large marketshare, and capabilities beyond anything in the open source space. Like:

  - partitioning (3 different types)

  - mature replication

  - high-availability solutions

  - parallelism

  - etc

So, you can spread a database across fifty commodity server (similar to a beowulf cluster) straight out of the box. You can also partition a table - and get massive performance-benefits on huge queries.

Since the open-source alternatives lack this commonly-used functionality, if you needed to manage vast amounts of data with an open source database your only option would be to spring a couple million for a huge regatta, E15k, etc - to try to get the same performance from mysql/postgresql/etc - that you could get from db2/oracle on a $50k server.

#

good article

Posted by: Anonymous Coward on December 21, 2004 02:32 AM
Thank you.

#

Firebird Evaluation

Posted by: Anonymous Coward on December 21, 2004 06:35 PM


The article is OK, but seems to damn Firebird with faint praise - whilst even Berkely db gets its own separate evaluation, all that's said about Firebird is this:


"Compare the failure of both Borland's Interbase and SAP's SAP-DB as open source projects -- no offense to Firebird DB, which became a dynamic project after they forked it away from Borland. "


That is just poor reporting. The article implies that Firebird is some "also-ran" open source project that was narrowly saved from extinction.


Firebird is a great db and can certainly stand comparison with Cloudscape, PostgreSQL and Ingres.
It is as featureful as any of those three. It has a 20 year history (as InterBase). It has a small footprint (embeddable) and is available on many different platforms (Win32, OS X, Linux, Solaris, HP UX). There are probably more client tools available for Firebird than for all the other open source databases combined. And compared to beasts like Oracle, the administration and maintenance of Firebird is a breeze (IBM is only just waking up to 'autonomic' databases, whilst from its inception InterBase and Firebird aimed at this).


And just to show that I am not so some savant in recognizing the value of Firebird....



  • In 2004 Firebird/InterBase won the O'Reilly database poll (http://www.oreillynet.com/pub/pq/46).

  • In 2003 it came neck and neck with MySQL, the winner of the Database of the Year Poll (403 votes vs 405 - http://www.linuxquestions.org/questions/showthrea<nobr>d<wbr></nobr> .php?s=&threadid=116360).

  • Even on Newsforge's own poll in December 2004, Firebird is the outstanding leader with 1638 votes to runner-up MySQL with 472 votes (http://www.newsforge.com/pollBooth.pl?qid=54&aid<nobr>=<wbr></nobr> -1).


Looks to me like the writer of this article is not aware of the massive surge in popularity and recognition that Firebird has received in the last two or three years. And all this despite the fact that even now a Google search on Firebird will still bring up more hits for the mis-named Mozilla browser than it will bring up for the database (and this situation was even worse in the past year).


Every developer in need of a database owes it to themself to consider Firebird. IBM gave me multiple copies of DB2 when I bought some other IBM software. I still have those unused licenses, becase DB2 does not offer me any more than Firebird (and would be a damn sight harder to install, use and maintain).


An excellent 1000+ page book on Firebird was recently published (you can get a copy from IBPhoenix and help support Firebird: http://www.ibphoenix.com/main.nfs?a=ibphoenix&s=1<nobr>1<wbr></nobr> 03619236:513792&page=ibp_firebird_book).


While you are there you can have a look at the range of administration and development tools available (http://www.ibphoenix.com/main.nfs?a=ibphoenix&pa<nobr>g<wbr></nobr> e=ibp_contrib_download).


I have no connection with IBPhoenix, or the Firebird development team. I am just a developer building a website using Firebird as the distributed database.

#

Re:Firebird Evaluation

Posted by: Anonymous Coward on December 22, 2004 04:56 AM
You quoted Josh as saying:

"Compare the failure of both Borland's Interbase and SAP's SAP-DB as open source projects -- no offense to Firebird DB, which became a dynamic project after they forked it away from Borland. "

I don't see the damning with faint praise here. I see Josh pointing out that Interbase floundered while Borland was attempting to maintain control. I then see him calling it a "dynamic project" which would hardly be damning with faint praise. If Josh had said something less complementary, then yes, I could see that.

The core PostgreSQL developers, like Josh, know the core Firebird developers, from meeting them at places like OSCON in the past, and have a great deal of respect for them, their database, and the effort they've put into it in the past.

I agree that Firebird is a great database, and I think it gets the short end of the stick when people are talking open source databases, as I'd place it way above MySQL in terms of quality and functionality, while MySQL Seems to get a lot of free press everytime they manage to implement some feature Firebird and PostgreSQL have had for years or even decades.

#

Re:Firebird Evaluation

Posted by: Anonymous Coward on December 23, 2004 12:15 AM
I was not criticising the PostgreSQL developer quoted - I was criticising the journalism in the article.

The article covers all the other major open-source dbs in roughly the same amount of detail, but only makes an allusion to Firebird by quoting that developer. The only reference to Firebird in that article is "no offense to Firebird DB, which became a dynamic project" - which does not convey any information about Firebird except to make it look like some kind of weak also-ran compared to the other DBs mentioned.



The polls suggest that it is other open-source DBs which are the also-rans... Also, by failing to say that Borland have continued to develop and invest in InterBase, the article implies that Borland abandoned InterBase once Firebird forked off from it. But my understanding is that Borland had a change of heart after opening up InterBase and decided that it was too valuable/lucrative to relinquish the revenue stream (unlike several of the other open-source DBs covered, InterBase was released under the extremely generous Mozilla open source license - if they had released it under a more closed license like that of MySQL then Borland could have continued to use it as a revenue stream). InterBase versions 6.5 and 7 are not open source, but it is still a successful, quality database engine.


I wrote my response to contribute some information so that others needn't waste their time looking at MySQL when they can use Firebird instead (I only found out about Firebird a couple of years ago in a footnote to a very lengthy debate about the pros and cons of PostgreSQL and MySQL). I didn't do it because I want to bash PostgreSQL (it has a few features I would like to see in Firebird, just as Firebird has some features I would like to see in PostgreSQL.) For all I know, the comments by the PostgreSQL developer were full of praise for Firebird, but were edited down to that off-hand comment.

#

Re:Firebird Evaluation

Posted by: Anonymous Coward on December 23, 2004 02:38 AM
> Every developer in need of a database owes it to themself to consider Firebird. IBM gave me multiple
> copies of DB2 when I bought some other IBM software. I still have those unused licenses, becase
> DB2 does not offer me any more than Firebird (and would be a damn sight harder to install, use and
> maintain).

I'd love a free database that offers me everything that Oracle & DB2 have. Freebird is definitely easier to manage than either of those two. It just can't handle the performance demands of either on the large end.

When it can resolve queries that scan 100 gbytes of data on a five-year-old machine in 1-3 seconds without using an index I'll check it out.

In the meanwhile, there are still a ton of smaller apps that could use it well I'm sure.

#

Re:Firebird Evaluation

Posted by: Anonymous Coward on December 28, 2004 05:44 AM
When it can resolve queries that scan 100 gbytes of data on a five-year-old machine in 1-3 seconds without using an index I'll check it out.




Wow, I don't know what kind of 5 year old machines you have that can do sequential scans at 33 to 100 gigabytes a second, but if you can spare one, ship it my way, will ya?



Name a commercial database that can do what you ask, I've never seen one, and I've used Oracle a fair bit. I think you're talking out the wrong end, myself.

#

Other open source database

Posted by: Anonymous Coward on December 23, 2004 02:43 AM
I'm not sure why the author missed the open-source database db.*. It's being promoted by a smaller company, ITTIA, and it fills a niche that the other players don't: very small footprint, duel network- and relational-model database. My experiences with it have been very positive, and it's worth a look.

#

Two years ago MySQL WAS a toy

Posted by: Anonymous Coward on December 23, 2004 05:14 PM
Interesting that Marten Mickos says "Two years ago, people said MySQL was a toy. Now, apparently everyone wants to be a toy!" when two years ago you had to use the very latest release of MySQL (4.0 at the time) if you wanted some of the fundamental features of enterprise databases such as;

- Transactions
- Multi-table deletes
- Automatic Crash Recovery
- Referental Integrity
- Row level locking
- Secure connection between database and client (using SSL)
- Query Caching
- Unions

I think it's more a case of MySQL has opened some doors that PostgreSQL didn't and thus the pair of them have shown commercial vendors that many open source databases can now compete on their terms.

#

Another name to consider

Posted by: uday_parmar on January 05, 2005 03:19 PM

Hi all,



Just thought that the readers of this thread could benefit from learning that <A HREF="http://www.daffodildb.com/" title="daffodildb.com">Daffodil Software</a daffodildb.com> is Open Sourcing One$DB, an edition of its popular Java RDBMS, Daffodil DB. This will happen sometimes in Jan 2005 itself. To see how Daffodil DB compares with other databases, have a look at this <A HREF="http://www.daffodildb.com/daffodil-comparison-sheet.html" title="daffodildb.com">Guide</a daffodildb.com>.



It is true that Open Source is here to stay in a big way. As an example, Daffodil DB has recently been made compatible with <A HREF="http://www.compiere.org/" title="compiere.org">Compiere</a compiere.org>, a leading Open Source ERP with an installation base of over 800,000. Till now, Compiere (being very demanding on databases), could run only on Oracle. Having received better-than-expected response from the Compiere community, I can now safely say that anyone with the right product can dethrone the biggies, albeit in a niche.



Uday Parmar

#

This story has been archived. Comments can no longer be posted.



 
Tableless layout Validate XHTML 1.0 Strict Validate CSS Powered by Xaraya