Creating a Search Engine
Tuesday 19 December 2006 8:10:00 pm
We have built a search interface to the new implementation. This installation includes the dataset of ez.no including the documentation but excluding the new bug system. The search interface is located at: http://eval.ez.no/content/searchevaluation
Searching
First you can limit the search to classes and / or sub-trees in the content hierarchy. This will define the context set for the search (important for calculating the document frequency on-the-fly).
Next, input the search query. These conditions are used for relevance-based ranking.
Unfortunately, in the current implementation, query performance is slow. It is not unusual for the search to take more than one minute. The lack of performance is mainly because of two reasons: First, there are different ranking algorithms which need to be processed. Second, the implementation was intended as a"proof-of-concept" and not a final implementation.
Rating of results
The list of search results combines the top 20 results of all algorithms. After the results are returned, you can provide relevance feedback by dragging and dropping the results to one of the boxes ("relevant", "partially relevant" and "not relevant").
Important: Please do not pick the relevant results only and move them to the "relevant" box. This is useless for our comparison of different algorithms. The best way is that you start with the first result, rate it by dragging and dropping it to the correct box and then go on with the next result.
When you are done (or when you don't want to rate any more results), you save the data by clicking save ratings. You will be redirected to the search result that shows your ratings so you can continue rating results if you want.
Thank you very much for your help. We welcome your comments and feedback.