The Saint George on a Bike project has launched a crowdsourcing campaign which seeks to train AI to automatically caption and classify paintings. Find out how you can get involved!
The Saint George on a Bike project aims to train and build AI systems which will be able to understand art and describe the visual content of paintings in detail. To make this work possible, the project has to train AI to recognise different elements in paintings; this is a potentially huge task, but also offers an opportunity to involve art lovers and cultural heritage professionals to contribute and get involved.
This is the motivation behind the project’s crowdsourcing campaign, which asks volunteers to submit annotations of digitised paintings. Volunteers are invited to create descriptions of 12th to 18th century European paintings on the campaign´s Zooniverse site. Volunteers can work on as many paintings as they want – the more descriptions gathered, the more data the AI algorithms can work with.
When annotating images for this campaign, you need to be as detailed as possible - as if you were explaining the painting to a person with their eyes closed. Phrases such as ‘small baby’ could be improved with detail - for example, ‘a small baby lies on a blue blanket on the ground’. This helps the AI to gain a much larger vocabulary that it can then use to automatically describe more paintings in the future.
Some people may wonder if it is necessary to describe everything they see in a painting. The general rule is that you should start with the elements that are important to understanding the image. It is not necessary to describe minor details such as the elements of architectural decor found in the scene. But don´t worry - the campaign has very clear instructions and examples about how to contribute. All you need is a willingness to help researchers improve their image captioning technology.
With thousands of paintings hosted on the campaign website, volunteers will also be able to view images from Europe’s vast artistic cultural heritage. Exploring different components and styles of centuries-old artworks could lead to new discoveries and engagement with art and cultural heritage.
Saint George on a Bike is an EU funded project led by the Barcelona Supercomputing Center in collaboration with Europeana. The project aims to improve the quality and quantity of open metadata associated with imagery from European cultural heritage, something which this crowdsourcing campaign supports.
This post was written collaboratively by colleagues from the Saint George on a Bike project.