Evaluating Peer-to-Peer Recommender Systems that Exploit Spontaneous Affinities

The validation of a recommender system is always a quite hazardous task, because of the difficulty of modeling the tastes of a given user. Novel (decentralized) recommender systems are proposed and evaluated by way of well known logs of user profiles and buddy tables, that contain lists of items with feedback ratings assigned by a given set of users. These information are cross linked, and the precision of the recommendation is compared with other well known (centralized) systems. This evaluation approach cannot be applied in the actual peer-to-peer domain: it is difficult, if not impossible, to build and maintain user profiles, and users are not required to give feedbacks to a data collector entity. Moreover, objects are poorly or not structured, and meta-information, when present, cannot be trusted because of fake files and incomplete item descriptions. In this paper, we present an evaluation process based on a 10-fold cross validation task, that we applied to estimate accuracy of the suggestions of a P2P recommender system recently proposed in [2]. The complexity of the evaluation of this peculiar recommender is increased because of "spontaneous affinities" between users that are used instead of classical knowledge representation based strategies.