MediaEval 2023

The MediaEval Multimedia Evaluation benchmark offers challenges in artificial intelligence for multimedia data. Participants address these challenges by creating algorithms for retrieval, analysis, and exploration. Solutions are systematically compared using a common evaluation procedure, making it possible to establish the state of the art and track progress. Our larger aim is to promote reproducible research that makes multimedia a positive force for society.

MediaEval goes beyond other benchmarks and data science challenges in that it also pursues a “Quest for Insight” (Q4I). With Q4I we push beyond only striving to improve evaluation scores to also working to achieve deeper understanding about the challenges. For example, properties of the data, strengths and weaknesses of particular types of approaches, and observations about the evaluation procedure.

The MediaEval 2023 Workshop will be held 1-2 February, collocated with MMM 2024 in Amsterdam, Netherlands and also online.

Detailed workshop schedule is in the Workshop Program

Task schedule

The MediaEval Coordination Committee (2023)

MediaEval is grateful for the support of ACM Special Interest Group on Multimedia

Task List

Medical Multimedia Task - Transparent Tracking of Spermatozoa

Participants detect and track sperm in microscope videos, using provided microscope videos and metadata. They must analyze both global and individual sperm attributes such as motility, speed, and distance traveled. The annotations follow WHO sperm quality standards and are verified by experts.

Read more.

Musti: Multimodal Understanding of Smells in Texts and Images

Participants develop classifiers to predict whether a text passage and an image evoke the same smell source or not and identify common smell sources text passages and images. Optionally, the challenge can be addressed in a cross-language setting.

Read more.

NewsImages: Connecting Text and Images

Participants are supplied with a large set of articles (including text body, and headlines) and the accompanying images from international publishers. The task requires participants to predict which image was used to accompany each article.

Read more.

Predicting Video Memorability

Participants automatically predict memorability scores for videos that reflect the probability that a video will be remembered. They will be provided with an extensive data set of videos with memorability annotations, related information, pre-extracted state-of-the-art visual features, and Electroencephalography (EEG) recordings.

Read more.

SportsVideo: Fine Grained Action Classification and Position Detection in Table Tennis and Swimming Videos

Participants address video analysis challenges in two sports: table tennis and swimming videos. Subtasks include position detection and action classification and involve leveraging different modalities in the video data: visual, sound, and text.

Read more.