direkt zum Inhalt springen

direkt zum Hauptnavigationsmenü

Sie sind hier

TU Berlin

Inhalt des Dokuments

Bachelor Thesis: A Web-based Hybrid Evaluation Scheme for Recommender Systems


A Web-based Hybrid Evaluation Scheme for Recommender Systems


An important component of the use of recommender systems is the need to evaluate the quality and value of these systems. The preferred method is to measure the accuracy of predictions in offline evaluations. But measuring accuracy alone does not allow to evaluate the quality of the recommender system as perceived by users. A single property cannot address all the objectives associated with the use of recommender systems. In addition to increasing the user satisfaction, these objectives include strengthening user loyalty and gaining a better understanding of the users' wishes. To achieve these goals, several other properties like novelty, diversity or serendipity are also important. But in most cases there are no hard measures to quantify these properties. In this thesis we use an approach that qualitative metrics from a user-centered online evaluation complement the results from the measurements of accuracy in offline evaluation. Especially in a university  environment a user study is necessary to get direct feedback from users and to evaluate non-accuracy metrics, as there is no access to a large database of real users that can be evaluated. Access to such databases is available to large companies which can directly survey their users on their satisfaction with the recommender system used. We therefore propose and develop a web-based hybrid evaluation scheme for recommender systems in order to combine quantitative and qualitative metrics. To measure the accuracy of recommender systems, we use cross-validations, a traditional method of offline evaluation. In addition, benchmark data sets with historical ratings from real users are used. In order to link the historical ratings from offline users with the ratings of participants in an online user study, a mapping from offline to online users is performed. In the web tool implemented by us, an online user rates and compares the recommendation lists that would actually have been calculated by recommender algorithms for an offline user. The online user then evaluates qualitative properties of these lists. This simulates direct feedback that is not available for the historical data. The ratings of offline and online users can thus be compared and linked with each other and different metrics can be examined. With our web-based hybrid evaluation scheme we show a way to complement quantitative metrics for accuracy with subjective qualitative metrics to get a more comprehensive picture of the quality and usefulness of recommender systems.  

Supervisor: Tobias Eichinger

Type:  Bachelor Thesis

Duration: 4 months

Zusatzinformationen / Extras

Quick Access:

Schnellnavigation zur Seite über Nummerneingabe

TU Berlin - Service-centric Networking - TEL 19
Ernst-Reuter-Platz 7
10587 Berlin, Germany
Phone: +49 30 8353 58811
Fax: +49 30 8353 58409