This website uses cookies to ensure you get the best experience. By clicking or navigating the site you agree to allow our collection of information through cookies. More info

Posted on Tuesday July 7, 2020

Updated on Monday July 20, 2020

Timematrix for researchers

Saint George on a Bike aims to improve the quality and quantity of open metadata associated with imagery from European cultural heritage. It especially aims to address the challenge of providing Artificial Intelligence with insights into culture, symbols and traditions. This webinar is aimed at academic researchers in data mining and data science that are interested in art and culture, and it will demonstrate results from the project and showcase the potential to adapt automatically produced descriptions of paintings to the time period when they were created. At the end of the course, a demo will be conducted to show correction of anachronisms and class refinement examples.

Google Map of Europa
Timematrix for researchers

Title: Image obtained from Saint George on a Bike

9 — 10 September 2020

In caption generator systems, the identification of the figures depicted in an image depends on a matrix of weights for each of the classes with which the system has been trained. Current caption generators are trained with images that reflect present time lifestyles. Therefore the matrix is in fact a TimeMatrix of the present. The course will show how the identification of classes varies as the matrix of weights depends on data relative to past centuries. This produces a time machine effect where the bike of a person in the TimeMatrix of the present becomes a horse of Saint George in the TimeMatrix of the 15th century.

Participants will be introduced to what we call the time machine effect, which consists of the objects of an image being transformed via deep learning methods to similar concepts that are more appropriate to another time period. The technical challenges and current solutions will be discussed.

Target group

Academic researchers in data mining and data science that are interested in art and culture.


Active interest in cultural heritage a must.
Basic knowledge of Natural Language Processing a plus.
Using Neural Networks a plus.

Course lecturers

  • Maria Cristina Marinescu (CASE Department, Barcelona Supercomputing Center)
  • Joaquim More Lopez (CASE Department, Barcelona Supercomputing Center)
  • Artem Reshetnikov (CASE Department, Barcelona Supercomputing Center)
  • Albin Larsson (Europeana)


Day 1. Understanding the past through its parallels with the present

9:00-9:15    Introduction to the general issue of describing cultural heritage images: what is done, and what is used for?
9:15-10:00    AI-powered semantic labelling of image datasets- formats, target concepts, challenges and description of TimeMatrix and its purpose
10:00-10:30    Training data in image datasets - images and metadata, challenges
10:30-10:40    Break
10:40-11:00    Present state-of-the-art caption generators and their limitations when dealing with anachronic images
11:00-11:30    Transfer learning to describe cultural heritage images as they differ from the present way of life - anachronisms, evolution of concepts through time, minorities, symbols
11:30-12:00    Ethical issues when describing works of art according to their time context

Day 2. Technical implementation of the TimeMatrix

9:00-9:15    Filtering anachronic classes
9:15-9:45    Defining new visual classes and visual relations. For  cultural and  iconographic symbols
9:45-10:15    Refining the identification of classes and visual relations according to time context
10:15-10:30    Deep Learning implementation of anachronism correction and class refinement
10:30-10:40    Break
10:40-11:00    Current state of SGoaB
11:00-11:30    Class features and Deep Learning implementation
11:30-12:00    Discussion of anachronic correction and class refinement examples (demo)

For more information, please consult the website
Please register for the event here