Last September, I wrote entry that became quite popular. In FreeBSD or Linux for your MySQL Server? I detailed some of the reasons that led me to recommend Linux over FreeBSD for MySQL servers at Yahoo. You may have also noticed my Solved: MySQL, FreeBSD, and LinuxThreads post back in October. Many people have e-mailed me recently to ask if the old posting is still accurate. A lot has changed since then and I'm writing this as a brief follow-up.
MySQL now runs very well on FreeBSD. I'm no longer steering people toward Linux. There are two important things you should do to make the FreeBSD/MySQL combo work well: (1) build MySQL with LinuxThreads rather than FreeBSD's native threads, and (2) use MySQL 4.x or newer. For more details, keep reading.
By building MySQL with LinuxThreads support (easy to do with the mysql40-server from the ports collection), you'll end up with a recent version of MySQL that doesn't suffer from any of the problems I described in my previous notes.
Many months back, one of Yahoo's uber-hackers contacted me to volunteer his time toward making MySQL run as well on FreeBSD as it does on Linux. After a few weeks time of tracking down a few bugs, we reached that goal.
The problem of MySQL threads not "noticing" that they were killed turned out to be bug in the FreeBSD kernel. That bug has been patched in FreeBSD 5.0. There's a work-around in the MySQL 4.0 source and beyond. It was never back-ported to 3.23, but it wouldn't be hard. Simply look for "Yahoo!" in the vio/vio.c file of MySQL's source tree to see what was necessary. I think the same bug was responsible for the wait_timeout not working correctly when using LinuxThreads.
The problem of MySQL occasionally thinking that all databases had vanished resulted from FreeBSD's non-threadsafe implementation of realpath(). The solution, as mentioned earlier, is to compile MySQL with the -DHAVE_BROKEN_REALPATH option. MySQL's configure script has been updated to do that automatically on FreeBSD systems. And a new implementation of realpath() was recently committed to FreeBSD's CVS repository.
About two weeks ago, I was contacted by Ian Failes (ianfailes@iprimus.com.au), who was conducting some e-mail based interviews about blogging. He asked me a series of questions and I responded. I asked him if he minded me posting them to my blog. He said he didn't mind, so here are his questions and my responses.
1. Why did you start a blog?
Jon Udell suggested it last year during a conference call. I no longer remember the details but it was somehow related to Linux Magazine. I'm good friends with Adam Goodman (the owner and publisher) and I've written for LM regularly for a while now. He got me and Jon on the phone to talk about blogs or something related to blogging.
2. Do you know how many people visit your site and blog, and what types of people they are?
I have a guess. I'd say that on a weekly basis, roughly 400 people read my blog (or parts of it). A lot more people read it on a less regular basis. I suspect that most of them are techies or friends of mine (or both).
3. What kinds of things do you put in your blog (ie. are they personal things or things you would like to share with others, or are they observations about the world you live in)?
I tend to blog about a variety of stuff. Some days it's technical tips about MySQL, other times it's a rant about work, still other times I'll just write to say I'm doing laundry. I do draw the line at some personal things, of course.
I started with the intent of blogging about techie stuff but quickly found it natural to write about whatever I felt like saying.
4. Why do you think bloggers are prepared to be personal in their blogs?
It's hard to say. Some people are just naturally open about themselves. Others like talking to a semi-unknown audience. They'll tell things that they don't even discuss with the close friends and family. I'm sure that some of them just want attention.
5. Why do you think readers of blogs read them?
Sometimes I wonder. :-)
Seriously, my friends read to find out what I'm up to. Everyone else probably reads because they find some percentage of what I say useful, interesting, entertaining, or otherwise worthwhile. At least that's my hope.
6. What kind of blogging sites, if any, do you like to visit? Why?
I subscribe to and read between 100 and 150 blogs every day. Many are other tech blogs, friends of mine, or people who've linked to me in the past. The blogroll on the right side of my blog home page is usually up-to-date.
7. Do you think that blogs will remain an individual and person thing or do you think it will become a way for companies or professionals to express themselves too, or is this already happening?
I'm not sure. I've been anxiously awaiting the evolution of "corporate blogging" but it's difficult to know how it will unfold. Jon has done a good job of tracking CxO (CIO, CEO, CTO, etc.) blogs. He ends up adding new ones to the "CxO bloggers" box on his blog when they surface. Right now they're mostly from tech companies. That's no surprise.
The big issue is whether or not corporate PR organizations are willing to the the CxO speak frankly and directly to the rest of the world. I suspect that most PR folks will fight it. Like many things, we'll need a critical mass of adoption to showcase the real benefits of corporate blogging.
8. Are there any problems that you foresee with blogging happening now or in the future?
There are many problems with blogging today, some technical and some social.
The technical problems center around making blog tools more accessible to non-techies and providing clean and simple mechanisms for cross-blog communication, like TrackBack. As time goes on, both will get better and better.
The social issues are a bit more interesting. What's safe to blog about and what's not? Why? Can you be fired for saying something bad about your employer or co-workers? Do you blog differently if you know your Mom is reading your posts?
I wrote about this a while back and got some interesting responses. Unfortunately, the best ones were via private e-mail and will remain that way. Suffice it to say, some companies and some families aren't as receptive to blogging as others.
9. What do you think blogging will be like in the next 2-5 years?
Blogging will have become mainstream. AOL and Yahoo will have blogging integrated (to some degree or another) in their services. Search engines may very well deal with blogs differently than they do now. Scott's Feedster is a preview of what's to come.
Expect to see blogging and annotations appear on popular news sites.
Beyond that, who knows. Those are the things I'm expecting to see. Maybe it'll become integrated with popular instant messaging systems too?
10. Is blogging just the modern version of 'dear diary'?
Not at all. Blogs, by nature, are both public and linked. Their power and popularity come from the human need to form communities. Some people think of their blog as an "on-line diary" but it's so much more than that.
The End
NetFlix sent me the two Bill & Ted movies recently. Why? They were on my list. I like watching stupid movies once in a while. They're rather entertaining to have on as background noise when I'm doing other stuff on a rainy Saturday.
Next up: Shadowlands. I don't remember who recommended it anymore. But that's the great thing about NetFlix. Every time someone suggests a movie I just stick it on my queue.
Wow, it's raining like mad right now. Very cool. I like rain (and thunder) storms in the spring. To bad we don't get many thunderstorms around these parts.
Why is today moving so slowly? It feels like I've been at work for 18 hours. And I'm tired. It doesn't feel like a Friday at all.
A couple weeks ago I took some friends flying and they brought along a kick ass little Sony DV camera. I borrowed the camera to suck the 44 minutes of video onto the TiBook and play with iMovie a bit.
After a visit to Fry's to buy a 120GB external FireWire/USB drive (I ran out of space), I had 8.5GB of video sitting on the Mac. So I messed around with iMovie long enough to figure out the basics (but not much more).
Just for fun, I've created a movie of a glider takeoff. I was flying the glider. Jeffrey Friedl ran the camera and his wife, Fumie, and her Mom were in the back seat of the glider.
I call this creation "Glider Takeoff" because that's exactly what
it is. Here it
is. It's a little big (16MB52MB), so if you're not on a high-speed
connection, don't bother. I'll do a low-bandwidth version if anyone
really wants to see it.
Don't go too hard on the filming technique or the my lack of iMovie skills.
I have over 44 minutes of video, so I'll try to do something a little more interesting for my next movie. Most of the film is in-cockpit flying, so that could be fun.
Update: My updload failed. The whole movie is not there. Fixing that now, but it'll take time.
Update #2: The whole movie is on-line now.
Well, a month and a half later the eagerly awaited (cough, cough, gag...) innovation contest winner was announced.
It amuses me greatly.
No, I'm not going to reveal the winning idea. That'd be a Bad Idea. I will say that I know someone who has been working to build a business around it for 2-3 years now and it's finally picking up some steam. But I won't spoil the fun and tell anyone at work that it's not really "innovative" if someone else is doing it (and has been for quite a while).
(Oh, wait. I think some of my coworkers read this.)
At least they found a way to get rid of that stereo. :-)
The -q option to mysqldump is poorly documented and misunderstood as a result.
Several folks at Yahoo have been having trouble with mysqldump running out of memory. At first I figured it was some odd problem, specific to one person's setup. But after it happened again, I started to look into it.
I did a bit of testing and found that when dumping really big tables, mysqldump was using up large amounts of memory. It was very good about freeing up the memory when it moved from table to table, but eventually it'd hit The Big One and die.
I reasoned that it must be buffering everything in RAM before dumping. (Of course, you can watch the long pauses in output to the dump file to notice this too.) I then thought that PHP has a way of fetching results without sucking tons of memory (mysql_unbuffered_query()) and began to wonder what it'd take to get mysqldump to do the same.
I ran thru the PHP code to figure out how they did it and found that it was the difference between calling mysql_store_result() and mysql_use_result() at the C API level. That's it, really. So I checked the source code to mysqldump only to find that the option already existed, but I'd have never found it.
The docs for mysqldump's -q flag say:
Don't buffer query, dump directly to stdout. Uses mysql_use_result() to do this.
Huh?
You see, unless you know the C API and understand the difference between buffered and unbuffered results from MySQL's point of view, the description in the manual is useless. I wouldn't have thought to try the -q option to solve this problem.
Ugh!
Sounds pretty trivial, doesn't it? It's not. The docs probably should say something about using it on larger tables and how you might run out of memory of you don't use it.
In fact, I'd go so far as to argue that maybe that option should be on by default. There's little downside for folks with small tables, and there's a lot of benefit for those with really big tables.
Anyway, this ends my public service announcement on the matter.
Following in the footsteps of Excite @Home and then AT&T, it appears that Comcast (or is it still AT&T? I can't keep track.) can't run a network either.
Matt's traceroute [v0.52]
cow Tue Apr 29 10:23:09 2003
Keys: D - Display mode R - Restart statistics Q - Quit
Packets Pings
Hostname %Loss Rcv Snt Last Best Avg Worst
1. cable-router.zawodny.com 0% 10 10 1 1 1 1
2. 10.150.152.1 20% 8 10 11 11 21 41
3. 12.244.98.193 20% 8 10 16 12 24 48
4. 12.244.67.17 10% 9 10 39 12 25 62
5. 12.244.72.210 0% 10 10 13 12 15 21
6. gbr5-p40.sffca.ip.att.net 10% 9 10 51 13 29 51
7. tbr1-p013501.sffca.ip.att.net 10% 9 10 21 13 36 69
8. ggr1-p340.sffca.ip.att.net 20% 8 10 14 13 23 57
9. svl-brdr-02.inet.qwest.net 20% 8 10 19 19 38 64
10. svl-core-03.inet.qwest.net 20% 8 10 56 20 30 56
11. chi-core-01.inet.qwest.net 30% 7 10 85 61 71 85
12. chi-edge-01.inet.qwest.net 10% 9 10 88 60 75 91
13. 65.112.65.14 10% 9 10 72 71 98 136
14. tlp3-atm1-0.toledo.oar.net 20% 8 10 82 80 88 112
15. wcnet-atm1-0s53.toledo.oar.net 0% 10 10 130 80 98 130
16. wcnet-fasteth3-0.toledo.oar.net 12% 8 10 84 81 91 129
17. family.zawodny.com 12% 8 10 91 84 105 144
Notice the packet loss at the first hop! I guess it's a good thing I held on to my DSL service too.
Sigh.
I just installed the new iTunes 4.0 and decided to check out Apple's new music store. I launched iTunes, clicked on the new "Music Store" icon and was taken to a web page that offered lots of stuff I might sample and buy. But there was one big, glaring problem with it.
Much of the music on that page is irrelevant to me. There's a wealth of information in my iTunes music library. Apple could use data about which music I own, how often I listen to various songs/artists/genres to make suggestions. You know, Amazon.com-style stuff.
I know why they don't do this by default (privacy), but the first time I visited, they really should have asked if they could use my music library and listening habits to better suggest music that I might want to try and possibly buy.
What are they thinking?
Amusingly, they even have a "Buy an iPod" link near the top of the page. Guess what, Apple. I own a 20GB iPod that's only 20% full. You could know this. I really have no need to buy another one right now.
There's so much potential here, yet so little of it is used.
On the plus side, at least Apple has the balls to try making the "pay for downloading music" model actually work. I give them a lot of credit for that.
The slides from my "MySQL, PHP, Stuff" talk on on-line now in HTML and PDF format.
Enjoy. (Well, as much as you can without the audio of me filling in all the detail.
Update: I'll fix the spelling bugs and re-post later today, I hope. Luckily my loyal readers are noting them in the comments.