MySQL to drop Berkeley DB storage engine, add memcache plugin


Author: Joe 'Zonker' Brockmeier

With the 5.1.12 release, MySQL will be dropping the Berkeley DB (BDB) storage engine from its list of supported database engines. Some observers have suggested that this move is connected to Oracle’s purchase of Sleepycat, and that InnoDB may be next. However, Brian Aker, MySQL’s director of architecture, says that they’re unlikely to drop InnoDB, and the Berkeley engine is being dropped for technical reasons. Aker also announced the first release of a memcache storage engine plugin for MySQL.

It’s easy to see how some might make a connection between Oracle’s purchase of Sleepycat and Innobase, which some saw as a move to disrupt MySQL’s business, and the decision to drop the BDB storage engine. However, Aker says that’s not the case.

If it’s not a move against Oracle, why drop BDB? For one thing, the BDB storage engine is not in widespread use. Aker says that he believes Debian is the only distributor of MySQL that ships with BDB enabled, and that “the main MySQL binaries have had it enabled in a very spotty manner.

“My belief is that I could count the number of installations on one hand. This should not infer that BDB was a bad engine, it has more to do with the fact that InnoDB has always had a more active developer group behind it.

“Active code in the open source world contributes to popularity.”

Rex Wang, vice president of embedded systems marketing with Oracle, says that the Berkeley DB team actually asked for the BDB engine to be removed. “To the best of our knowledge, there are few, if any, MySQL deployments that use Berkeley DB today, so we believe there is no impact on MySQL’s and Oracle’s joint customers.”

However, even though BDB is not widely used, Aker says that it won’t disappear from MySQL 5.0 or previous releases, so users who have deployed MySQL 4.x or MySQL 5.0.x shouldn’t worry if they are using BDB. Aker also noted that “MySQL has never offered commercial support for BDB, so nothing changes there either.”

MySQL hasn’t made a lot of noise about the decision to drop BDB, but it was noted in the release notes for 5.1.12. The 5.1 series is considered beta, and 5.1.12 has not been released yet, so it’s not surprising that MySQL has not made a formal announcement.

Blogger Peter Laursen complained that this was an example of poor communication on MySQL’s part, and suggested that InnoDB may be next.

Aker says that dropping BDB is not a precursor to the removal of InnoDB, and there’s no reason to assume MySQL will dump the InnoDB storage engine because of its purchase by Oracle.

My dog is also paranoid that every squirrel might be a burglar. Last week I spent several hours working on the interface to InnoDB cleaning up the code. I see bug fixes continue to roll in from Heikki. A few months ago I spoke to Ken Jacobs about how to extend the MySQL SQL syntax to leverage fulltext support from InnoDB. I see no signs that InnoDB is going away or that Oracle will stop supporting it.

And why would MySQL drop InnoDB? It’s used by a significant number of our users. I use it for websites, so I have a personal interest in it continuing to work well. 🙂

Wang indicates that Oracle will continue to support the InnoDB storage engine for MySQL. “InnoDB is by far the dominant transactional storage engine for MySQL, and Oracle plans to continue to provide that to MySQL, including ongoing enhancements and full support. So there’s really no reason for MySQL and Oracle to support Berkeley DB as a second transactional storage engine.”

Wang notes that BDB was the first transactional storage engine for MySQL, but InnoDB rapidly became the most popular choice once it was available. “This was because InnoDB was dedicated to supporting the needs of MySQL as the only user, while Berkeley DB was focused on meeting the needs of a very broad customer base and community. MySQL used a modified version of Berkeley DB, and the MySQL driver was never properly optimized to use it. No work was done on this interface for many years. In contrast, InnoDB has been continuously optimized for use with MySQL, and it is quite mature and proven.”

Plugins, anyone?

Still, the benefit of open source is that anyone can add support if they truly want it. If the community is interested in continued support for BDB, Aker says that BDB could be added easily as a plugin storage engine, and pointed to the PrimeBase XT transactional engine for MySQL and Solid Information Technology’s storage engine for MySQL as other storage engines added as plugins rather than part of MySQL itself.

Aker also mentioned the memcache storage engine plugin that he has been working on, which will add memcached functionality directly to MySQL, rather than running memcached separately.

Memcached is used extensively by high-traffic sites such as LiveJournal to boost performance by allowing clients to access information in memory rather than going to disk. Aker says that using memcache as a storage engine, rather than using memcached separately, will make life easier for developers.

“The cost of round trips drops because the data can be sent from within the database. You can also compare memcache contents from within the database using simple selects. Ease of use, integration, less round trips … it all adds up to being a good idea.”

The plugin is still in early development. Aker refers to it as “pre-alpha,” so some SQL syntax will probably not work with the memcache storage engine yet. Aker says that SELECT, UPDATE, DELETE, and INSERT should work, but ORDER BY operations, REPLACE, and a few other operations are unlikely to work.

Is it likely that the memcache storage engine will become part of mainstream MySQL? Aker says it “depends on response and usage. Today it’s faster to develop the memcache engine outside of the main MySQL tree. I can certainly see a world where we would decide to incorporate it.”