Automatic Evaluation of World Wide Web Search Services
By Abdur Chowdhury, America Online Inc., and Ian Soboroff, National Institute of Standards and Technology, USA. 2002. The authors present a method for comparing search engines automatically based on how they rank known item search results. The queries are constructed using query log analysis, the result is automatically constructed via analysis of editor comments from ODP. Five search services are compared: Lycos, Netscape, Fast, Google, HotBot, with the result that some services perform better than others, but the majority are statistically equivalent.