sphinx search If you use the Sphinx search engine and have been watching the development branch (0.9.10) and wondering when to upgrade, I'm here to tell you that "now" is a great time. As of r2037, the last major issue I regularly saw has been fixed. The other big bug was fixed in r2031.

Late last week I began testing those fixes in a "burn-in" test I've developed that makes liberal use of indextool --check. Instead of seeing index corruption within an hour, I saw none. After 3 days of no failures, I deployed it to a subset of our search back-end servers. Yesterday we deployed it to half of the remaining servers.

So far, so good!

I should note that all our index corruption was merge related. Sphinx wasn't building corrupt indexes out of the box, but the merges (usually filtering merges) could produce corrupted indexes.

We were upgrading from a lightly patched version of r1894. That meant rebuilding our indexes to use the new and more compact format. Some of the obvious benefits of the upgrade:

  • smaller disk and memory footprint
  • pre-fork support to spawn searchd children at start up
  • more reliable shutdown and pid file handling
  • kill lists
  • mysql protocol support
  • lots of small optimizations and fixes

Thanks to the Sphinx team for their excellent work. I look forward to the release of Sphinx 1.0.

Posted by jzawodn at October 30, 2009 07:26 AM

Reader Comments
# said:

Thanks for your update. I wrote a review of the previous release and was worried when I read about some of the bugs on this one.

I can't wait to test 1.0 either.

on October 30, 2009 09:53 AM
# Mengembalikan Jati Diri Bangsa said:

I've never heard Sphinx search engine before. I'm gonna try this now. Thanks for sharing this

on November 2, 2009 12:39 AM
# Neeraj said:

I am also using the net since 10 years and Sphinx search is new to me i found about it through your post only.

on November 8, 2009 01:18 PM
# Business Solutions said:

I 've heard about Sphinx before. But never used it thinking that its not necessary or worth. Reading your article ( this one and your previous one ) I think it can do something with business. Let me give it a try. Thanks for sharing it.

PS: If your blog is having a quick - easy subscription button that will be great.


on November 10, 2009 01:24 AM
# Todd Atkins, LCSW said:

I've have to check this out. Thanks.

on November 10, 2009 08:27 PM
# Dog Life Jackets said:

Thanks for your post i would like to have regular visit to it

on November 11, 2009 09:56 AM
# pudge said:

We've been having a lot of sphinx performance issues in sphinx 0.99-rc1. I think searchd's forking is a big reason why. I am going to give the latest a try and see what happens.

Is there somewhere a document outlining how forking works? What the searchd child does on startup, when it starts up, how long it lives ... ?

on November 19, 2009 01:17 PM
# Bustor said:

Sphinx like other search engines is overshadowed by the great Google.
But I think that people should go use other search engines too. If their usage increases then the one sided equation of Google could change and needless to mention there is a vast possibility of huge knowledge data banks

on December 8, 2009 04:34 AM
# monster energy hats said:

Fantastic website I will bookmark it and come back later.
Thanks for posting this. Very nice recap of some of the key points in my talk. I hope you and your readers find it useful! Thanks again

on May 4, 2010 03:38 AM
Disclaimer: The opinions expressed here are mine and mine alone. My current, past, or previous employers are not responsible for what I write here, the comments left by others, or the photos I may share. If you have questions, please contact me. Also, I am not a journalist or reporter. Don't "pitch" me.


Privacy: I do not share or publish the email addresses or IP addresses of anyone posting a comment here without consent. However, I do reserve the right to remove comments that are spammy, off-topic, or otherwise unsuitable based on my comment policy. In a few cases, I may leave spammy comments but remove any URLs they contain.