IIIF for 3D – making web interoperability multi-dimensional

IIIF standards and 3D models

IIIF (short for International Image Interoperability Framework) is a set of open standards, a community and a consortium of cultural institutions. It is increasingly adopted as institutional repositories seek to open up their collections and share resources across institutional silos.

But support for 3D remains a yet-to-be-resolved issue. There is the need for specifying a third dimension (i.e. a Z-axis) over the 2D-coordinates (i.e. the X-, Y-axis) of the ‘canvas’ space that so far has been able to serve the model of IIIF for 2D and audiovisual file viewing and annotating. Handling lighting scenarios, camera perspectives and the possibility to interact with multiple models within a single viewer frame require further specification and agreement among stakeholders.

The recent event DaSCHCon 2023 brought together perspectives on the topic from research data repositories, tool makers and institutional collections, alongside a workshop of the Technical Specification Group (TSG) for 3D in the IIIF-consortium.

The conference featured presentations from the IIIF Technical Coordinator and the co-chairs of the Technical Specification Group for 3D. The Technical Specification Group shared the main use cases for the specification and opened the space for discussion and further clarification of scope. The use cases ranged from core functions, such as being able to display a single model at a particular position, scale and orientation within the 3D space, to displaying one or more 3D models alongside 2D and AV media, commentary and more. A new concept within the IIIF spec – a ‘scene’ – was proposed to serve the use cases by establishing the following assumptions:

• A scene is a boundless space with a right-handed coordinate system;

• The Y-axis indicates upward movement and the Z-axis indicates forward movement;

• Canvases can be nested within scenes; and scenes can also be nested within other scenes;

• A model is placed within a scene by means of an annotation with motivation specified as ‘painting’. Text-based labels or commentry can be applied to a model (or a scene) by means of annotations with other motivation types.

More challenging aspects regarding the specification of types, positions and directions of lights and cameras were discussed in the course of the conference.

Viewers and libraries

Tools to render 3D models visible on the web (referred here as ‘viewers’) play an important role in specifying the IIIF standards, because the standards have to be easy to implement in order to have broad adoption. The conference featured presentations about specific tools such as MorphoSource and Semantic Kompakkt.

MorphoSource is an open source project from Duke University, with a large and well-established collection of 3D models and user contributors. It is based on the Aleph 3D viewer extension and supports a wide range of file formats, as well as annotations.

Semantic Kompakkt is the tool developed at TIB’s Open Science Lab as part of the NFDI4Culture initiative. It is based on the Kompakkt viewer (which itself is using the babylon.js library) and combines Kompakkt’s collaborative annotation features with semantic web capabilities thanks to an integration with Wikibase. Kompakkt was originally conceived as an open source alternative to Sketchfab, the popular commercial platform.

Other tools and libraries relevant to defining IIIF spec parameters, include: Blender (as important 3D data generation and pre-processing tool), Mozilla Hubs (for bringing together multiple models in a single space), 3DHOP (for selecting of polygon regions for annotation), and three.js (for parameter references).

Repositories and data processing workflows

Presentations from DaSCH and the University of Basel focused on concrete data workflows and how IIIF can play a role in bringing together archival and usability concerns. A key takeaway: preparing your media for better interoperability is the best strategy to also secure long-term preservation.

Tool pipelines for optimising the processing of large 3D media included open source tools, such as MeshLab, Gimp and Blender, with potential for future automation via Python scripts. The Smithsonian Cook was cited as a successful example of an automated data pre-processing pipeline.

Institutional experience from a 3D digitisation project was also shared by the ETH Library and their Collection of Scientific Instruments and Teaching Aids. The project has so far made more use of commercial tools in their workflow, including publishing the collection on Sketchfab. Sketchfab was selected because it reaches a wide public audience, with tracked visitor numbers and downloads showing impact, and it offers direct integration with Europeana.eu. A further step towards interoperability would be to re-publish their collection objects on open viewers such as MorphoSource and Semantic Kompakkt – a prospect discussed during the Q&A.

Outlook

Tools like Sketchfab and open source alternatives like MorphoSource and Kompakkt make it possible to upload and view single 3D models on the web with different options for collaboration and annotation. More complex environments like Mozilla Hubs offer the potential of positioning multiple objects in a 3D scene in relation to each other. Staying true to the IIIF Design Principles (to make simple things easy and complex things possible), the 3D specification would need to ensure that viewers can both implement a ‘minimum viable product’ for user interaction and remain interoperable, as well as include more complex interactions and still specify those in a standards-compliant way.

Get involved

Are you interested in 3D interoperability? Here’s how to get involved:

• Join the IIIF community to learn more and consult with other community members if you need help with your digitisation workflow.

• Join technical group meetings and test implementations.

• Report issues or use case feedback related to IIIF on Github.

• Publish your data on one or more online tools and repositories, and consider what reasons you might have to go with a commercial or open source environment.

To discuss IIIF and other topics related to research and development in the cultural heritage sector, you can also join the EuropeanaTech community.

Help shape the next phase of sharing multi-dimensional cultural heritage on the web!