|Welcome Guest ( Log In | Register )||Resend Validation Email|
Posted: Jul 24 2011, 03:57 PM
Like an eagle!
Group: SITE OWNER
Member No.: 1
Joined: Sep 15th 2002
Location: San Leandro, California
As a selected few of you have already know, the search engine for the forums was upgraded to Full-Text Search. As the number of posts slowly accumulate through the years, we now have over 1.14 million posts on the forums. Every time when one conducts a search, the search engine has to look through ALL these posts to find matches. It is both time-consuming and resource-intensive.
The old algorithm was straight forward, and that is scanning through every posts one by one. The size of all 1.14 mllion posts together is around 700MB. Even with MySQL's optimized search algorithm, a regular search took anywhere between 10 seconds to 60 seconds. What's worse was whenever someone search, the whole forums locked up for several seconds to several tens of seconds. In my opinion, that is just unacceptable. With the wealth of information we have to our disposal, searching for information should be painless and quick. This is where the new Full-Text Search algorithm shines. The Full-Text Search indexes all the posts and pick out keywords and put them in a pool of keyword list and associate each keyword with the correspond Post ID. When someone conduct a search, it quickly looks through this pool of words and find the ones that matches what the person is looking for. It then returns the Post ID. The whole process (minus the indexing) takes only a fraction of a second.
(Note: There is also a Simple Mode to this upgrade, but it is not that big of a difference. The only difference is the way it returns the results. In Simple Mode, there are two options: Most Relevant and Most Recent.)
The downside to Full-Text Search is maintaining the index, which adds another 300MB to the database. I believe the benefits of such upgraded search engine completely outweigh the downside. There are new ways to search what you want easier and more precise. You can search a phrase now by putting quotes around the phrase. (e.g. "Fifth Stage") You can also specify which word you are looking for in a post by simply putting a '+' sign in front of the word and putting a '-' sign in front of the word to exclude it in a post. Wildcard is also possible by putting '*' before or after the word. The possibility is endless. For more information on Full-Text Search, please visit the official MySQL page: http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html
If you have any question regarding this upgrade, please either PM me or start a topic in Feedback and Support, thanks!
Proud Contributor of the Music Section Revival Project