Overview of newsreel’16: Multi-dimensional evaluation of real-time stream-recommendation algorithms

Benjamin Kille, Andreas Lommatzsch, Gebrekirstos G Gebremeskel, Frank Hopfgartner, Martha Larson, Jonas Seiler, Davide Malagoli, András Serény, Torben Brodt, Arjen P De Vries

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

11 Citations (Scopus)


Successful news recommendation requires facing the challenges of dynamic item sets, contextual item relevance, and of fulfilling non-functional requirements, such as response time. The CLEF NewsREEL challenge is a campaign-style evaluation lab allowing participants to tackle news recommendation and to optimize and evaluate their recommender algorithms both online and offline. In this paper, we summarize the objectives and challenges of NewsREEL 2016. We cover two contrasting perspectives on the challenge: that of the operator (the business providing recommendations) and that of the challenge participant (the researchers developing recommender algorithms). In the intersection of these perspectives, new insights can be gained on how to effectively evaluate real-time stream recommendation algorithms.
Original languageEnglish
Title of host publicationExperimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2016
Subtitle of host publicationInternational Conference of the Cross-Language Evaluation Forum for European Languages
Place of PublicationCham
Number of pages21
ISBN (Electronic)978-3-319-44564-9
ISBN (Print)978-3-319-44563-2
Publication statusPublished - 23 Aug 2016
EventCLEF 2016 - Conference and Labs of the Evaluation Forum: Information Access Evaluation meets Multilinguality, and Interaction - University of Évora, Évora, Portugal
Duration: 5 Sep 20168 Sep 2016
Conference number: 7th

Publication series

NameLecture Notes in Computer Science
ISSN (Electronic)0302-9743


ConferenceCLEF 2016 - Conference and Labs of the Evaluation Forum
Internet address


  • Recommender Systems
  • News
  • Multi-dimensional Evaluation
  • Living Lab
  • Stream-based Recommender


Dive into the research topics of 'Overview of newsreel’16: Multi-dimensional evaluation of real-time stream-recommendation algorithms'. Together they form a unique fingerprint.

Cite this