Flexibility for the future: Europeana software now hosted by cloud infrastructure
Imagine a software engineer spending all his time writing software code. Sounds natural, right? Well, over the last few years, this has become pretty far from the reality of the software industry for most engineers. Particularly in small and medium software companies, they routinely do a whole load of tasks not mentioned in their job descriptions: installing and upgrading software, managing pieces of IT infrastructure, increasing and decreasing storage space and more.
This state of affairs is the result of deeply-rooted trends in the technology world. Large-scale monolithic software systems developed by organic teams continuously move towards mash-ups, built up out of reusable components. In-house development of new products and features often gives way to integration of third party services and tools. Businesses engage more in outsourcing, subcontracting and other forms of work carried out with remote teams and service providers.
Fluctuating demand is another source for this operational flexibility: a marketing campaign leading to a thousand-fold increase in web traffic can require a substantial overhaul of the system. A piece of software perfectly capable of serving thousands users a day can crash under the load of millions.
The notorious ‘devops’ euphemism conceals a grim new reality: software developers (dev) are increasingly expected to work as software operators (ops). Their attitudes differ: some fret, with good reason, that they were hired to write code and not format disks. Effectiveness suffers because specialisation, the hallmark of knowledge economy, is compromised.
A hot-air balloon flies over a park. The Wellcome Library CC BY.
Platform-as-a-Service (PaaS)
But there is hope. Along with many other promises, cloud computing has one which aims to address the growing ops part the devops package. It is called Platform-as-a-Service (PaaS) and it intends to reverse the trend by farming out the operations part of the developer work. PaaS systems are sophisticated software hosting solutions which provide software operation as a service.
For example, a PaaS system can automatically scale up an application in heavy demand, sparing manual intervention. Or, it can provide a ready-made managed service for a database, so its installs and upgrades are done as part of the service.
Europeana has first-hand experience of the woes of ops. For starters, we must, as many other service providers on Internet, adjust to fluctuations in demand. For example, a prime-time feature on Europeana1914-1918 on German television resulted in a huge surge in traffic to Europeana websites.
Flexibility for our Network
But there are strategic reasons to address operations seriously. As a network organization, we work extensively with many technical partners across Europe. We and our partners are often involved in integration projects, perform large-scale experiments with our data, try out new technologies or carry out provisionary installations to be used by a project or a partner. Such tasks fall mainly into the ops part of devops. Performing them with efficiency and flexibility is key to our long-term success as a network.
To address the growing need for efficiency in operations we decided to take advantage of the PaaS approach. Last year we ran a full-scale European Tender to choose a new infrastructure provider. In this tender a big part of the requirements related to providing operations of the infrastructure as a service. After a scrupulous evaluation and scoring process we had a winner – Anynines, powered by Avarteq hosting company from Saarbrücken, Germany. Anynines are using CloudFoundry, a popular open-source PaaS solution.
The comfort of PaaS systems does not come for free as they only support software compatible with certain technological principles, designed to increase its interoperability. After the tender, as part of the migration process, we spent a considerable amount of time making Europeana software compatible with CloudFoundry. By the end of February we were happy to make the final switch and raise a toast to Europeana’s success.
We are still far from fully realizing the potential of the PaaS approach. We still need to make some components fully compatible with it, to get know the system better, and, most importantly, to start exploring how our technical partners can fully take advantage of it.
As part of the Europeana Digital Service Infrastructure, our vision is to have a flexible hosting solution for cultural applications working with Europeana and developed by our network partners. No less important than the ease and comfort of our joint ops, we also envisage an infrastructure services bill lower than the sum of bills we pay today. Stay tuned for future updates, and let us know your thoughts.