The MediaEval Multimedia Evaluation benchmark offers challenges in artificial intelligence for multimedia data. Participants address these challenges by creating algorithms for retrieval, analysis, and exploration. Solutions are systematically compared using a common evaluation procedure, making it possible to establish the state of the art and track progress. Our larger aim is to promote reproducible research that makes multimedia a positive force for society.
MediaEval goes beyond other benchmarks and data science challenges in that it also pursues a “Quest for Insight” (Q4I). With Q4I we push beyond only striving to improve evaluation scores to also working to achieve deeper understanding about the challenges. For example, properties of the data, strengths and weaknesses of particular types of approaches, and observations about the evaluation procedure.
The MediaEval 2023 Workshop will be held 1-2 February, collocated with MMM 2024 in Amsterdam, Netherlands and also online.
Detailed workshop schedule is in the Workshop Program
MediaEval is grateful for the support of ACM Special Interest Group on Multimedia
Participants detect and track sperm in microscope videos, using provided microscope videos and metadata. They must analyze both global and individual sperm attributes such as motility, speed, and distance traveled. The annotations follow WHO sperm quality standards and are verified by experts.Read more.
Participants develop classifiers to predict whether a text passage and an image evoke the same smell source or not and identify common smell sources text passages and images. Optionally, the challenge can be addressed in a cross-language setting.Read more.
Participants are supplied with a large set of articles (including text body, and headlines) and the accompanying images from international publishers. The task requires participants to predict which image was used to accompany each article.Read more.
Participants automatically predict memorability scores for videos that reflect the probability that a video will be remembered. They will be provided with an extensive data set of videos with memorability annotations, related information, pre-extracted state-of-the-art visual features, and Electroencephalography (EEG) recordings.Read more.
Participants address video analysis challenges in two sports: table tennis and swimming videos. Subtasks include position detection and action classification and involve leveraging different modalities in the video data: visual, sound, and text.Read more.