Interesting things are afoot in the MySQL world. You see, it used to be that the MySQL world consisted of about 20-40 employees of MySQL AB (this funny distributed Swedish company that built and supported the open source MySQL database server), a tiny handful of MySQL mailing lists, and large databases were counted in gigabytes not terabytes. A Pentium III was still a decent server. Replication was a new feature!

Hey, anyone remember the Gemini storage engine? :-)

How times have changed...

Nowadays MySQL is sort of a universe onto itself. There are multiple storage engines (though MyISAM and InnoDB are still the popular ones), version 5.1 is out (finally), and the whole company made it over 400 employees before it was gobbled up by Sun Microsystems (a smart move, IMHO, though history will judge that) a while back.

If I had to guess 5 years or so ago what would be interesting to me today about MySQL, I'd have been really, really wrong. The future rarely turns out like we think. Just ask Hillary Clinton.

Here's a little of what's rattling around in the MySQL part of my little brain these days...

Outside Support, Patches, and Forks

The single most interesting and surprising thing to me is both the number and necessity of third-party patches for enhancing various aspects of MySQL and InnoDB. Companies like Percona, Google, Proven Scaling, Prime Base Technologies, and Open Query are all doing so in one way or another.

On the one hand, it's excellent validation of the Open Source model. Thanks to reasonable licensing, companies other than Sun/MySQL are able to enhance and fix the software and give their changes back to the world.

Some organizations are providing just patches. Others, like Percona are providing their own binaries--effectively forks of MySQL/InnoDB. Taking things a step further, the OurDelta project aims to aggregate these third party patches and provide source and binaries for various platforms. In essences, you can get a "better" MySQL than the one Sun/MySQL gives you today. For free.

Meanwhile, development on InnoDB continues. Oh, did I mention the part where they were bought by Oracle (yes, *that* Oracle) a while back? Crazy shit, I tell you. But it makes sense if you squint right.

Anyway, the vibe I'm getting is that folks are frustrated because there's not a lot of communication coming out of the InnoDB development team these days. I can't personally verify that. It's been years since I corresponded with Heikki Tuuri (the creator of InnoDB). So folks like Mark Callaghan of Google have been busy analyzing and patching it to scale better for their needs.

And we all benefit.

Drizzle

Taking things a step further yet, the Drizzle project is a re-making of MySQL started primarily by Brian Aker, who worked as MySQL's Director of Architecture for years. Brian is now at Sun and, along with a handful of others at Sun and elsewhere, is ripping out a lot of the stuff in a fork of MySQL that doesn't get used much, needlessly complicated the code, or is simply no longer needed.

In essence, they're taking a hard look at MySQL and asking what it really needs to provide for a lot of it's uses today: Web and "cloud" stuff. He visited us at Craigslist a few months ago to talk about the project a bit and get our input and feedback. I believe it was that day I joined one of the mailing list and started following what's going on. Heck, I even build Drizzle on an Atom-powered MSI Wind PC regularly.

It's great to see a re-think of MySQL going on... keeping the good, getting rid of the bad, and modularizing the stuff that people often want to do differently (authentication, for example).

It's even better to see the group that's hacking on it. They really have their heads on straight.

Unanswered Questions

Why is all this even necessary? Are the "enterprise" customers and their demands taking focus away from what used to be the core use and users of MySQL? Is Sun hard to work with?

It's clear that both the MySQL and InnoDB teams could be doing more to help. But having worked at a large company for long enough, I realize that things are rarely as simple as they should be.

Will this stuff get integrated back into mainline MySQL? Will Linux distributions like Ubuntu, Debian, and Red Hat pick up OurDelta builds? What about Drizzle?

Will Drizzle hit its target and be the sleek and lean database kernel that MySQL once could have been?

Hard to say.

It's hard to guess what the future holds and too easy to play armchair quarterback about the work of others. But these are question worth wondering about a bit.

What's it all mean?

Nowadays MySQL has a much slower release cycle that it used to. It's still available in "commecial" and free ("community") releases. There's still a company behind it--a much larger one in fact. But one that also has a vested interest in showing how it works better on their storage appliances or 256 "core" computers and whatnot.

Clustering is still very niche. Transactions are not.

Meanwhile, all the cutting edge stuff (at least from the point of view of scaling) is happening outside Sun/MySQL and being integrated by OurDelta and even Drizzle. The OutDelta builds are gaining steam quickly and Drizzle is shaping up.

Heck, I'm hoping to get an OurDelta box or two on-line at work sometime soon. And I'd like to put a Drizzle node up too. I want to see how the InnoDB patches help and also play with the InnoDB plug-in (and its page compression).

The next few years are proving to be far more interesting than I might have expected from a project and technology that looked like was on a track straight for Open Source maturity.

And you know what? I like it.

Posted by jzawodn at December 08, 2008 07:02 AM

Reader Comments
# John Robinson said:

I've been keeping my eye on the development of Drizzle as it progresses. I believe that it will eventually be something I'd like to use on my server. I'll be watching the progress over the next few months.

Great article BTW.

on December 8, 2008 08:51 AM
# Owen Byrne said:

Interesting post. There's an incomplete sentence near the end though - "There's a growing network of"...

on December 8, 2008 09:00 AM
# Jeremy Zawodny said:

Owen: Thanks. Fixed the dangling sentence bits.

on December 8, 2008 09:26 AM
# Aurimas Mikalauskas said:

Thanks, Jeremgy - a beautiful overview of waves around MySQL and the community.

P.S. A small typo - Web and "could" stuff.

on December 8, 2008 01:48 PM
# Jeremy Zawodny said:

Fixed. Thanks!

on December 8, 2008 01:51 PM
# Egor Egorov said:

Heh, I remember being official support and consulting partner of MySQL AB, diging everyday thru the mailing list :)

I think the market is always in the need of small and light database. There was mSQL. Then there was MySQL. Now MySQL is a beast and there's a clear need to tailor it down to a light server. Maybe that will be Drizzle.

Personally, being extremely loyal to MySQL server I now feel like I am looking for other opportunities in my projects. Drizzle is far from production and MySQL is going to get 6.0 into GA full of fat way before Drizzle will become usable in mass-scale web sites...

Interesting times we live :)

on December 8, 2008 04:17 PM
# Joe said:

Another small typo.

The OutDelta builds are gaining steam quickly and Drizzle is shaping up.

Good write up, thank you.

on December 8, 2008 10:36 PM
# Chris said:

Damn, I'm getting old, I had no idea you left y!

on December 9, 2008 01:37 AM
# Jan said:

Hi Jeremy,

maybe you should look for an editor :)

"Sweedish" -> Swedish.

on December 9, 2008 03:45 AM
# Jeremy Zawodny said:

Jan: and give you guys nothing to complain about? Never! :-)

on December 9, 2008 07:30 AM
# Arjen Lentz said:

Antony has been hacking on the original Gemini engine codebase, it's tentatively called Amira and it's had a number of fixes and performance tweaks.
We might at some point provide it as a 5.1 plugin for OurDelta.

on December 9, 2008 03:25 PM
# Ken Jacobs said:

Hi Jeremy ... I'd like to update you w.r.t. to the position of the InnoDB team on incorporating community contributions. Instead of repeating my note here, I'll just point to the comment I logged on Kevin Burton's blog, where the issue first arose: http://feedblog.org/2008/12/04/destroying-mysql/#comment-12456.

Hope that helps clarify things somewhat!

on December 9, 2008 03:42 PM
# Jeremy Zawodny said:

Ken:

Thanks for stopping by to point out that comment.

on December 9, 2008 03:44 PM
# Ken Jacobs said:

You bet, Jeremy. Glad to do so.

Re your comment that some folks would like to see more communication from the InnoDB team. All I can say is "stay tuned". We have heard that concern and will be responding. For now, please visit our forums (http://forums.innodb.com).

Thanks.

Ken

on December 9, 2008 03:58 PM
# WOW GOLD said:

nice post!
-http://www.vcsale.com/ buy wow gold

on December 9, 2008 10:10 PM
# ray said:

Hi,

I was reading ur blog posts and found some of them to be very good.. u write well.. Why don't you popularize it more.. ur posts on ur blog ‘The New MySQL Landscape’ took my particular attention as some of them are interesting topics of mine too;

BTW I help out some ex-IIMA guys who with another batch mate run www.rambhai.com where you can post links to your most loved blog-posts. Rambhai was the chaiwala at IIMA and it is a site where users can themselves share links to blog posts etc and other can find and vote on them. The best make it to the homepage!

This way you can reach out to rambhai readers some of whom could become your ardent fans.. who knows.. :)

Cheers,

on December 12, 2008 04:11 AM
# more typos said:

Seeing as everyone else is doing it: usres of MySQL

on December 15, 2008 08:41 PM
# Log Buffer said:

"Jeremy Zawodny surveyed this new MySQL landscape, full as it now is of patches, forks, and Drizzle, and liked what he saw."

Log Buffer #128

on December 23, 2008 08:57 AM
# Mike said:

Well I think that this was great.

on March 5, 2009 12:57 PM
# JacksonV said:

IBM and Sun Microsystems both make computer systems for the corporate world. By purchasing Sun, IBM would get a leg up in the global finance and telecommunications markets. Representatives from both sides have yet to comment as of this writing. Sun Microsystems is both information technology and software company, and been recognized since the 80s. They had become one of the biggest competitors with Microsoft for IT and corporate software and hardware. However, after the dot com bust in the early 2000s, Sun Microsystems has begun to struggle a bit. Instant payday loans aren't really going to help them, but they have entered negotiations with IBM for a buyout that's supposed to total around $6.5 billion. News of the talks has boosted Suns' stock on the market. It may be the best move for them to avoid staring down bankruptcy, which would be a disaster for a firm as large as Sun Microsystems.
CLICK THE LINK FOR MORE DETAILS:
http://personalmoneystore.com/moneyblog/2009/03/18/ibm-sun-microsystems-buyout/

on March 24, 2009 02:05 AM
# Joel said:

I hear that. I remember the days when it was hard to catch a release announcement before release. These days there's a much longer wait. Depth of quality or depth of bureaucracy? I'm not sure.

Joel

on March 30, 2009 08:47 AM
# gaming said:

both the number and necessity of third-party patches

on June 25, 2009 06:46 PM
# finance expert said:

All the time I tried to understand what is MySQL. Found you blog now and I very hope that I learn it carefully.
Thank you for your work.

on November 18, 2009 11:13 PM
# life insurance said:

I feel a lot more people need to read this, very good info!
life insurance(http://www.ratedetective.com.au/insurance/life-insurance)

on January 21, 2010 11:21 PM
# real estate programs said:

Interesting read, thanks for helping keep me busy at work ;)

real estate programs(http://www.realestateit.com.au/)

on January 21, 2010 11:25 PM
# full tilt said:

A good read, definitely worth a cut and paste. Thanks!

full tilt

on January 21, 2010 11:27 PM
# mejor lcd said:

MySQL is great , first of all it is free and second it is fast and reliable.

on April 28, 2010 11:15 PM
# metin2 yang said:

very good post

on July 15, 2010 12:26 AM
Disclaimer: The opinions expressed here are mine and mine alone. My current, past, or previous employers are not responsible for what I write here, the comments left by others, or the photos I may share. If you have questions, please contact me. Also, I am not a journalist or reporter. Don't "pitch" me.

 

Privacy: I do not share or publish the email addresses or IP addresses of anyone posting a comment here without consent. However, I do reserve the right to remove comments that are spammy, off-topic, or otherwise unsuitable based on my comment policy. In a few cases, I may leave spammy comments but remove any URLs they contain.