Who's Using What?

Who's Using What is a blog series by Gregory Markus from The Netherlands Institute for Sound and Vision as part of EuropeanaTech. The idea behind the series was to raise awareness about the brilliant open source software options now available to institutions, and to encourage collaboration in the digital heritage community. What better way to show off these tools than talk to the developers making use of and developing them? You can also find lots of OS options in the new EuropeanaTech FLOSS Inventory.

For this issue of Who's Using What EuropeanaTech spoke with Bret Davidson and Jason Casden from North Carolina State University Library. We were drawn to Bret and Jason because of this fascinating paper they had published in the Code4Lib Journal. The paper, "Beyond Open Source: Evaluating the Community Availability of Software" where they discuss extending the open source software values of collaboration and transparency to include the wide and affordable distribution of software as a way of increasing the efficient deployment, uptake and continued use of the rich Open Source tools available for libraries. They investigate whether larger, better financed institutions have a responsibility to assist smaller instituions with the implementation of certain OS tools as this will by and large imporve the community as a whole. Bret and Jason's answers are detailed and insightful. A wonderful addition to the growing Who's Using What catalogue.

Still from a video about the Automated Book Delivery System or 'bookBot' at James B. Hunt Jr. Library, North Carolina State University

1. What open source tools are you currently working with?

Much of our current work is focusing on using virtualization, container, and provisioning technologies like Vagrant, Docker, and Ansible, to improve the availability of our software to less well-resourced institutions or standardize our development and production environments. We are also looking at opportunities to support open research practices and research training using tools like Jupyter, the successor to iPython notebooks, and developing broadly applicable strategies for social media harvesting and archiving. We are exploring the capabilities of Solr 5 for supporting statistical reporting and data science needs in our applications, Capistrano and DevOps best practices for simplifying production application deployment, Vagrant for unifying our development environments across developers, and persistence.js and localForage as solutions for persistent in-browser data storage. Of course, we are also constantly evaluating a number of text editors like vim, Atom, and Sublime Text, but we can’t seem to come to any agreement on which one is best.

2. What open source tools have you used in the past to develop larger applications?

We’ve used a huge variety of open source tools, libraries, and languages over the years. Most of our web applications are written in either Ruby or PHP with JavaScript for client-side work (and a bit of server-side code using node.js), and we frequently use Python for basic server scripting and quick data analysis tasks. We try to use established frameworks with thriving user communities where possible, like Ruby on Rails, Zend, and Angular.js, to improve the quality and stability of our applications and increase the amount of time we can invest toward solving our primary goal of supporting new and enhanced library services. We’ve stuck with mature databases as well, focusing primarily on MySQL with some projects using PostgreSQL where needed. We’ve also used Solr extensively to meet many of our indexing and searching needs. That said, we have experimented with cutting edge libraries and technology where it makes sense in our projects, like persistent in-browser storage for supporting offline data collection and D3.js for developing rich data analysis tools. In these cases, we typically share the results of our investigations with the broader library community.

3. What are you currently developing?

We’ve been making progress recently on using Vagrant, Docker, and Ansible to package social media archival software like Social Feed Manager (developed at George Washington University) and Lentil (developed at NCSU) into a single software package, Social Media Combine, that can be deployed quickly and easily to desktop computers and used to start initial social media archiving programs across the spectrum of cultural heritage institutions. This system was supported by a grant from the State Library of North Carolina funding the creation of a Social Media Archives Toolkit to guide archivists in developing their own social media collection programs.

Recently, we’ve started the process of migrating the data collection client of our mobile space assessment toolkit, Suma, to Angular.js and integrating updated in-browser storage tools like localForage to improve the cross-browser compatibility of the application. If that all goes well, we hope to add some features that have been requested by our user community. Suma is a leading open source tool that has been adopted by over 100 institutions around the world, but we’ve found from user feedback that institutions with less IT resources often struggle to implement Suma, irrespective of the level of documentation. To address this, we have been working on developing Docker containers and Vagrant development environments for Suma to make it easier for our users to deploy Suma to a wider variety of environments and lower the barrier to entry for developers who want to contribute to Suma.

4. What would you like to see developed?

We think the library community would greatly benefit from the existence of more sophisticated software packaging tools (similar to those available to the developers of desktop applications) for server-based software, with a particular focus on making server-based software available to individuals and smaller institutions. We have presented some potential guidelines for evaluating the effectiveness of our software distributions], and the Hydra-In-A-Box project is also undertaking promising explorations in this area.

We have started receiving an increasing number of requests from researchers for relatively straightforward research data management and collaboration platforms (i.e. problems that could be mostly solved using Microsoft Access), but the cost of repeatedly developing these systems is fairly high relative to the number and consistency of the requests. There are commercial application and database builders, such as Knack and Doolli, that allow less-technical users to create and manage relational data stores, and we think that an open-source version of this idea would be extremely useful if hosted by libraries.

Maybe the most significant thing we would like to see isn’t necessarily a tool or library, but a community around public services library technology. Although individual open source software projects often have their own communities and there are excellent organizations around library technology and infrastructure like LITA and Code4Lib, we think there is still an opportunity for developers and designers of library public services software to come together to collaborate on open source software.