Jeremy Zawodny's blog

Is MySQL 5.1 a compelling upgrade?

Of the many things I noticed last week at the MySQL Conference, one of the most notable was how many companies have not upgraded from MySQL 5.0 to 5.1 yet. Craigslist is in that camp and it seems that we're joined by the likes of Facebook, Google, Yahoo, and about half a dozen other companies that use MySQL heavily.

Come to think of it, SmugMug are the only folks I've talked with who've made the jump (video).

So it's not much of a surprise that Percona is asking if they should backport 5.4 fixes to 5.0.

Given our usage of MySQL to date, the only really compelling reason to upgrade is to get access to the InnoDB plug-in (and XtraDB). I'd like to get compression, some of the various performance patches, and tuning options, so plug-in support is a requirement. But beyond that, I just don't see anything new in 5.1 that we need.

As I noted in The Real or Official MySQL? Does Not Matter!, the storage engines matter more than the various add-on features in the server itself.

Have you upgraded or are you thinking about it? If so, why? If not, why not?

MySQL and Drizzle Tip: Checking configuration file syntax (faking configtest)

In the Apache world, you might be familiar with tweaking your config file(s) and then running

$ apachectl configtest

to see if the config parses. We've been discussing this on the drizzle mailing list and talking in general about configuration handling and management. Well, it turns out that you can fake it in MySQL and Drizzle too.

If you have a new configuration in /tmp/new.cnf, try this:

$ mysqld --defaults-file=/tmp/new.cnf --verbose --help

And it'll run mysqld (or drizzled), parse the config, report any problems, print help, and exit without initializing storage engines or trying to grab a port.

Neat trick!

Thanks to Baron Schwartz, Arjen Lentz, and Sheeri Cabral (book) for helping to demonstrate this.

Slides from "MySQL and Search at Craigslist"

Last week I delivered a talk titled "MySQL and Search at Craigslist" as part of the 2009 MySQL Conference and Expo. I talked about some of the good and bad of our MySQL work and also talked a lot about our recent Sphinx deployment. The slides are embedded below and here, thanks to SlideShare. (Anyone know why Google Docs doesn't yet handle OpenOffice presentations?)

I gave a copy to O'Reilly but don't yet see them on the conference site.

The usual disclaimers apply: I said a lot that's not well reflected in the slides, and I'm sure they're less informative without the audio or video that may or may not have been captured. Either way, hopefully they're useful to folks who saw the talk and even a few of those who did not.

I also delivered a condensed version of this talk at the Percona Performance Conference and those slides are available too.

Thanks to everyone who provided useful feedback and discussion before and after the talks.

Hack on Drizzle Full-Time for Rackspace!

Given the current state of the economy, here's a quick job plug for anyone interested and qualified.

At the Drizzle Developer Day on Friday, I got to meet Adrian Otto from Rackspace. Rackspace has a cloud offering (think Aamazon EC2) that's called Mosso and is willing to employ full time developers who spend all their time working on Drizzle.

Here's what he sent to the mailing list.

I was speaking with Eric Day at the developer conference, and I mentioned that Rackspace is wiling to employ full time developers for the specific purpose of furthering the Drizzle project's mission. He suggested that I email you on this list becuase he expected there would be interest in this offer. If you work on the project now part time, and want to make it a full time job working exclusively on the Drizzle project, let me know. The Rackspcae Cloud believes in open source, and we want to do our part to make Drizzle a wild success.

Talking with him a bit, the rationale is simple: Rackspace wants to offer the best cloud resources they can. Part of that means having infrastructure that their customers need and works well. They're betting the Drizzle is part of their future, and hiring a few people to work on it makes that future a reality sooner than later.

It looks like Mark Callaghan (Google) likes the idea too, as does Don MacAskill (SmugMug).

Anyway, ping me if you're interested and I'll put you in touch.

Slides from What Craigslist wants and needs from Drizzle

As I previously mentioned, on Friday I attended the Drizzle Developer Day at Sun in Santa Clara. While there I had the chance to speak to the group while everyone ate their salad, pizza, and cookies.

The talk was titles "What Craigslist wants and needs from Drizzle" and is available as a Google Docs presentation here. I've also embedded a version of the slides below.

I should note here, as I did at the talk, that this presentation is neither comprehensive or completely representative. That is to say that I'm sure there are things I've forgotten. Plus, the fact that I was working with MySQL in other high-volume web shops before coming to Craiglist means that there's definitely some personal bias and pet peeves addressed in there too.

Anyway, that's what I presented.

Thanks to the fine folks at Sun (soon to be Oracle) for hosting and organizing the day. And special thanks to the Drizzle developers for getting together and showing the rest of us how things work and taking time to talk about their plans.

MySQL and Percona Conferences Rocked

Now that it's Friday, I can finally come up for air and say that both the MySQL and Percona Conferences (which I wrote about earlier) surpassed my expectations. Having the two going on semi-concurrently worked out pretty well. At no time did I find myself without at least two sessions I wanted to see. Often times I had to really cut lunch short to make sure I didn't miss anything.

Other MySQL Conference veterans I asked has very similar responses.

There's a ton of other stuff to digest, and I hope to write up some thoughts in the coming days and weeks.

As usual, a big part of the conference for me was being able to meet up with people I don't see often or who I've never met face to face. Meeting up with people solving similar problems at places like Facebook, Google, Mixi, and other high-traffic sites is invaluable.

Being able get a good sense of what new storage engines are out there and how modern hardware is changing database systems (SSDs, multi-core servers) proved to be very educational in ways I didn't expect.

All the politics and posturing around the Oracle/Sun buyout weren't as significant as I'd expected. People really wanted to get down to business and technology.

I enjoyed giving my talks and answering questions about problems that we have and haven't solved yet at Craigslist.

I'm heading over to Drizzle Devloper Day in a couple hours.

MySQL and Percona Performance Conferece Lineup

Amidst all the Oracle/Sun/MySQL news today, the MySQL Conference kicks off this week. So I just spent a few minutes putting together my picks for the sessions I'd like to attend at the MySQL Conference and the Percona Performance Conference (schedule). There's quite a lineup and I have some hard choices to make. Both groups have put together excellent events. And, wow, there are a lot of new storage engines and appliances coming out.

To make my life easier, I'm putting the list of interesting sessions from both conferences here so I can try to decide where to spend my attention.

Sessions I'm Presenting or Part Of

I'm presenting MySQL and Search at Craigslist on Tuesday morning and am part of The Great Open Cloud Shootout on Wednesday morning.

On Friday, I'll be at the Drizzle Developer Day to talk about "What Craigslist Needs from Drizzle."

Tuesday Sessions: MySQL

• 8:30am - State of the Dolphin - Karen Padir
• 9:15am - This is Not a Web App: MySQL at Google - Mark Callaghan
• 10:50am - MySQL and Search at Craigslist - Jeremy Zawodny
• 11:55am - InnoDB: Innovative Technologies for Performance and Data Protection - Ken Jacobs, Heikki Tuuri
• 2:00pm - Falcon Storage Engine - Designed for Speed - Kevin Lewis, Ann Harrison
• 3:05pm - The PBXT Storage Engine: Meeting Future Challenges - Paul McCullagh
• 4:25pm - Solving Common SQL Problems with SeqEngine - Beat Vontobel
• 5:15pm - Hadoop and MySQL: Friends with Benefits - Frank Mashraqi

Wednesday Sessions: MySQL

• 8:50am - The Great Open Cloud Shootout - panel
• 10:50am - Build Your Own MySQL Time Machine - Chuck Bell, Mats Kindahl
• 11:55am - Using Q4M: A Message Queue Storage Engine for MySQL - Kazuho Oku
• 11:55am - libdrizzle: A New Client Library for Drizzle and MySQL - Eric Day
• 11:55am - libdrizzle: A New Client Library for Drizzle and MySQL - Eric Day
• 2:00pm - Maria: The New Transactional Storage Engine for MySQL - Monty Widenius
• 2:00pm - SAN Performance on a Internal Disk Budget: The Coming SSD Revolution - Matthew Yonkovit
• 2:00pm - Crash Recovery and Media Recovery in InnoDB - Heikki Tuuri
• 3:05pm - MySQL Performance on EC2 - Mark Callaghan
• 4:25pm - Perl Stored Procedures for MySQL - Antony Curtis
• 4:25pm - High Availability and Scalability Patches from Google - Ben Handy, Justin Tolmer
• 5:15pm - Optimizing MySQL Performance for ZFS - Allan Packer, Neelakanth Nadgir
• 5:15pm - Redundant Storage Cluster: For When It's Just Too Big - Bob Burgess
• 5:15pm - Inserts at Drive Speed: Designing a Custom Storage Engine for Write-Mostly Applications - Ben Haley

Wednesday Sessions: Percona

• 9:00am - Maria In Depth - Monty Widenius
• 9:55am - The Return of Gearman - Eric Day
• 11:15am - Fighting Replication Lag - Peter Zaitsev
• 12:45pm - Evaluating Disk Backends for MySQL Servers - Ewen Fortune
• 1:35pm - Database Performance with Proxy Architectures - Robert Hodges
• 2:00pm - Covering Indexes: Orders-of-Magnitude Improvements - Dr. Bradley C. Kuszmaul
• 5:00pm - Sphinx and MySQL: A Perfect Match - Andrew Aksyonoff
• 6:15pm - InnoDB Performance Tuning - Peter Zaitsev
• 7:05pm - CouchDB: Behind the Buzz - Jan Lehnardt
• 7:55pm - Linux Filesystems: Who, What, and Where - Stewart Smith
• 9:10pm - Open Q&A: Performance - panel

Thursday Sessions: MySQL

• 8:30am - The SmugMug Tale - Don MacAskill
• 10:50am - SQL is Dead - Monty Taylor
• 10:50am - Map/Reduce and Queues for MySQL Using Gearman - Eric Day, Brian Aker
• 10:50am - Dormando's Proxy for MySQL - Alan Kasindorf
• 11:55am - Memory Management in MySQL and Drizzle - Stewart Smith
• 11:55am - Improving Performance by Running MySQL Multiple Times - MC Brown
• 2:00pm - MySQL Row Change Event Extraction and Publish - Gene Pang
• 2:00pm - InnoDB Performance and Usability Patches - Vadim Tkachenko, Ewen Fortune
• 2:50pm - Make Your Life Easier with Maatkit - Baron Schwartz
• 2:50pm - BLOB Streaming: Efficient Reliable BLOB Handling for all Storage Engines - Barry Leslie
• 3:50pm - Database We Can Believe In: Stories from the Front Lines (of the Obama Campaign) - many speakers

Thursday Sessions: Percona

• 9:55am - Pushing the Envelope - Don MacAskill
• 10:50am - The Life of a Dirty Page - Mark Callaghan
• 1:35pm - High Performance MySQL from a Boring Architecture - Baron Schwartz
• 3:15pm - Hypertable - Doug Judd
• 4:30pm - Drizzle's Approach to Improving Performance of the Server - Jay Pipes
• 7:50pm - MySQL Replication: Getting The Most From Slaves - Peter Zaitsev
• 9:05pm - Open Q&A: Feature Request Bonanza - panel

Friday: Drizzle Developer Day

Drizzle Developer Day is on Friday at Sun in Santa Clara. I'm looking forward to many of the talks (some of which will be completely over my head but interesting anyway).

Back To Work!

And, with that, I need to go work on my presentation! See you at the conference... I'll try to post interesting tidbits on Twitter and tag 'em with #MySQLConf like others appear to be doing.

I just found out that I can share my public MySQL Conference Schedule.

Oracle Buying Sun, Gets MySQL

Interesting news this morning, just as the 2009 MySQL Conference is starting. As is being reported all over the place, Oracle has agreed to buy Sun at $9.50 per share, giving them to a ton of great technology (Solaris, ZFS, MySQL, DTrace, etc.).

One one of the biggest threats to Oracle's core database business (at the low end, at least) for a while now has been MySQL. And now they're poised to own MySQL after Sun bought it not long ago. (It seems like yesterday that Oracle bought Innobase.)

As I noted a while back, the MySQL landscape is changing.

This news is sure to make the conference more... interesting.

Oracle, please get the InnoDB team together with the MySQL team and see about GPLing ZFS.

Sponsor Our Ride For Diabetes (Tour de Cure 2009)

In early May, Kathleen and I will be participating it the Tour de Cure 2009, a bike ride to raise awareness and money for Diabetes. Craigslist (my employer) is sponsoring a team that we'll both be riding on. Collectively, our team is trying to raise $75,000 during this years ride.

If you have a few bucks to spare for a good cause, please consider sponsoring me or sponsoring my wife (or both!). It's for a very good cause.

We're both riding the 25 mile course and would love even a $1/mile contribution. As a bonus, Craigslist is matching all our donations. So if you donate $25, your contribution becomes $50 thanks to the company's generosity.

Here are links for a bit more information:

• Our Team Page
• Kathleen's Page
• My Page

You can visit either of our pages to pledge on-line. And if you're interested in riding, visit our team page.

Thanks for any support you can offer!