Idomaar: A Framework for Multi-dimensional Benchmarking of Recommender Algorithms

Mario Scriminaci, Andreas Lommatzsch, Benjamin Kille, Frank Hopfgartner, Martha Larson, Davide Malagoli, András Serény, Till Plumbaum

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

1 Citation (Scopus)

Abstract

In real-world scenarios, recommenders face non-functional requirementsof technical nature and must handle dynamic data in the formof sequential streams. Evaluation of recommender systems musttake these issues into account in order to be maximally informative.In this paper, we present Idomaar—a framework that enables theefficient multi-dimensional benchmarking of recommender algorithms.Idomaar goes beyond current academic research practicesby creating a realistic evaluation environment and computing botheffectiveness and technical metrics for stream-based as well as setbasedevaluation. A scenario focussing on “research to prototypingto productization” cycle at a company illustrates Idomaar’s potential.We show that Idomaar simplifies testing with varying configurationsand supports flexible integration of different data.
Original languageEnglish
Title of host publicationPoster Proceedings of ACM RecSys 2016
Subtitle of host publicationProceedings of the Poster Track of the 10th ACM Conference on Recommender Systems, RecSys 2016
EditorsIdo Guy, Amit Sharma
Place of PublicationAachen
PublisherCEUR
Pages1-2
Number of pages2
Publication statusPublished - Sep 2016
Event10th ACM Conference on Recommender Systems, RecSys 2016 - MIT, Boston, MA, United States
Duration: 15 Sep 201619 Sep 2016
https://recsys.acm.org/recsys16/

Publication series

NameCEUR Workshop Proceedings
Volume1688
ISSN (Print)1613-0073

Conference

Conference10th ACM Conference on Recommender Systems, RecSys 2016
CountryUnited States
CityBoston, MA
Period15/09/1619/09/16
Internet address

Fingerprint

Dive into the research topics of 'Idomaar: A Framework for Multi-dimensional Benchmarking of Recommender Algorithms'. Together they form a unique fingerprint.

Cite this