In one sentence, what’s Phase two about?
Phase two focuses on the practical side of measuring the changes - the outcomes - that are experienced by your stakeholders as a result of your activities.
What will I do?
You’ll build your data collection plan by extending your Change Pathway from Phase one. You’ll collect the data you need to show you if an outcome, the change, has taken place for your stakeholders. You’ll analyse and interpret the data to emerge with your findings.
Can I follow all parts of Phase two myself?
We first published Phase two as a PDF in early 2020. Learning from how others have used it and how they respond to the topic of impact measurement, we know that Phase two - impact measurement can be considered the most tricky phase of all. Over time, we realised that there was more to cover, and we knew that we had to work extra hard to make this phase easy to understand for heritage professionals. At different points we ask you to evaluate if you have the right skills internally and to consider if you might need external help.
What parts of the Impact Playbook have you redesigned to help us on our impact assessment journey?
Based on what we’ve learned from the Impact Community, we’ve expanded a few areas that are key to the impact assessment and data collection process.
1. Developing indicators
An indicator is information that allows us to measure whether you are achieving your desired outcomes. It shows whether the outcome has happened or not. Each outcome might have one or many indicators, and these can be a mix of subjective (more qualitative) or objective (more quantitative) measures.
There are a number of existing indicator types that you could use, or you could develop your own. We’re working to make more standardised indicators that we can use in the Europeana Initiative. In the interim, and drawing on what we’ve learnt by collecting data extensively over the last four years, we developed the Standardised Question Bank.
2. Sample sizes and response rates
Another area that we’ve expanded is guidance on sample sizes and response rates.
Your sample sizes and response rates determine whether your results are going to be representative of the group that you are surveying. It’s unlikely that you’ll get a 100% response rate, nor that you can be 100% confident that if you were to ask a different group the same questions, that the results would be the same. The group is likely to be diverse in terms of its characteristics, like gender, background and perspectives. How might this change how you ask them questions and what questions you ask?
3. Methods
We’ve grown the list of methods that we explored in Phase two to include more digital methods and more arts-based methods (which has seen a surge of interest in the last years). We also include methods that help you assess your environmental impact and we’ve expanded the section on economic impact assessment methods.
4. Ethics and data protection
We want everyone who is using the Impact Playbook to collect data only when they are confident about ethics and data protection. This isn’t as hard as it seems, though it does take some thinking, planning and research to make sure that you are following good practice and legal guidelines.
How do I find out more?
The Impact Playbook is now online for you to use in your work.
