How to create a VR game using Europeana.eu

The Immersive Tech Week Hackathon is a three-day hackathon held at the yearly Immersive Tech Week in Rotterdam, the Netherlands. Participating students accept one of the many challenges posed by industry leaders, non-profit organisations and public institutions and develop a project that solves a specific problem or use case, using immersive technologies.

Students Zizi, Mats, Jasmijn, Daan, Steven, Thomas and Fatih are studying Minor Digital Humans at Rotterdam Hogeschool, and were encouraged by their teachers to take part in the hackathon. From over 10 challenges, this team of students decided to work on the challenge posed by Europeana: Showcase an immersive way of storytelling in a 3D environment with real-time interactions to experience history and/or culture, using content from europeana.eu.

So, how did the team approach the challenge?

Step 1: Ideation and planning

The three-day hackathon was described as a whirlwind of creativity and collaboration. Coming from diverse backgrounds, the group found joy in working intensely together, forming a strong bond and gaining a deeper understanding of each other. Despite the challenges of managing numerous tasks, they described the experience as both hectic and enjoyable.

'The hackathon was really hectic, we had to tackle a bunch of different tasks at once, but this also really made us bond together as a group. You really get to know each other when you spend three days together non-stop!' said Jasmijn Oorschot. 

After brainstorming and researching, the team identified that they wanted to create something which would:

• Excite and engage people through European history and culture

• Appeal to younger generations by creating a gamified immersive environment as a learning experience

• Acknowledge that cultural heritage institutions often grapple with their contentious pasts, legacies of looted objects and colonial structures.

The team came up with a game that tackled the issues at the intersection of these strands. 'Heritage Heist,' is an XR game you can play using a VR headset or by using an AR app on your phone. The game revolves around the issue of stolen artefacts, raising awareness about the provenance of many objects while also being a fun and engaging learning experience.

Step 2: Assets, file formats, platforms, and coding

Once the game concept was decided on, the team started searching Europeana.eu for 3D objects to put in their virtual museum (3D objects being appropriate for placing in a virtual space to create an immersive environment, which was one of the prerequisites of the hackathon). They found statues of the god Toth (anthropomorphised as a baboon) and Alexander the Great, that they could download in .obj format. They then complemented their selection of objects with 3D cultural heritage artefacts from other websites, like this Plateosaurus skeleton from the Natural History Museum in Vienna.

'We learned a lot about the artefacts we used in our project, because we really had to research those artefacts ourselve,' says Mats Verlinden.

Initially the students wanted to use Blender, a program they were familiar with, to construct the virtual space the game would take place in. They quickly realised that the files they had didn't seem to function well with either Blender or Unity (another programme). After a tip from another mentor, they started building their game in Fectar. Fectar is a platform that allows anyone to easily create 3D content that can be accessed through a variety of devices: using your phone's camera to project in AR, through a VR headset, or with a web browser. With Fectar you can import, place, design and animate 3D models to create a fully immersive 3D experience.

This meant that the group had to familiarise themselves with an application they'd never used before, but luckily Fectar was easy enough to learn that they got the hang of it within their timeframe of three days. See the two Europeana objects come alive in Fectar in the video below:

One of the main hurdles the group faced was that 3D models digitised by cultural heritage institutions often are very detailed and thus very large. These models weren't made for reuse in a game that needs to load and run quickly and smoothly, but for research and study where every minute detail is highly important. It's clear when an object gets digitised by a cultural heritage institution, the institution might have a very specific reuse case in mind that makes other types of reuse more difficult. Especially for 3D models, one model might be suitable for 3D printing, but not for reuse in an immersive VR environment, or for research and preservation.

Step 3: finalising and presenting

No challenge was too big for the Heritage Heist theme, who finished their game in time for the presentation on the third day of the Hackathon. They even created a trailer for the game using AI tools: Dall-E 3 for static artwork, ChatGPT for the script, RunwayNL for video content, and Elevenlabs for the voice-over. The result of this work is the trailer you can see below.

For their proof of concept the Heritage Heist team won first place in the 'Best Solution to the Challenge' category. They perfectly answered Europeana's challenge by creating a creative immersive experience that was not only fun, but also tackled relevant social dilemmas.

Find out more

You can visit the virtual museum where the Heritage Heist takes place through Fectar!

Are you inspired to create a project using Europeana data? Then get started with our API documentation or send us a message with your idea at api@europeana.eu.