A Story About Impact
Creating an Impact Measurement Framework for SJOG Hospitaller Services
Type of work: Research, design and implementation of a new framework
Duration: 1 year for first version - Ongoing
Team: SJOG Hospitaller Services UK - Opportunities team
My strengths: planning and execution, leading the project, involving stakeholders
Impact: - Better understanding and use of data collected in charity at different levels
- First impact report published for 2024
- Aligning of perspectives on what impact means for the charity and their strategy
The Impact Measurement Project is a strategic initiative aimed at developing and implementing a system to capture and showcase the impact of SJOG Hospitaller Services UK across its four main areas of work: homelessness, modern-day slavery and trafficking, disabilities, and older religious communities. Evidencing better what the charity does was part of their 2024 strategy.
This project addresses the challenge of evidencing the charity's multifaceted work in a consistent and comprehensive manner, with the goal of effectively communicating their achievements to various stakeholders, including the people they serve, commissioners, funders, and the wider public.
As the Impact Lead, my role has been to design and implement this framework across the charity to ensure we accurately reflect the positive changes SJOG brings to the communities it serves.
The process of creating SJOG’s impact framework
To create SJOG's impact framework, I employed the double diamond approach, iterating through cycles to develop a robust theory of change model and identify key indicators that effectively demonstrate SJOG's impact.
Discover
-
Conducted research on impact measurement methodologies across other organisations and literature/theory.
-
Identified need to set clear goals and missions for each of SJGO’s service area.
-
Defined what impact means for the people working at SJOG.
-
Chose the theory of change model to structure the framework as research pointed to this model as a good starting point.
-
The theory of change can ensure accountability, guide service delivery, address operational challenges, and align activities with SJOG's strategy and social care sector perspectives and changes.
Define
Defined the theory of change for five areas of care: disabilities, complex needs, modern day slavery and trafficking, homelessness, and older religious communities by:
-
Engaging with service managers and team leaders to gain insights into processes and operations, and linking inputs and activities to expected outputs and outcomes.
-
Researching national perspectives and statistics to align with global needs, and setting clear, mission-driven goals for each service area.
-
Identifying common outputs and outcomes across services, and balancing standardised reporting with respect for individual service variations and individual needs and outcomes for the people supported.
-
Delimiting impact evaluation to three lenses: personal, operational, and strategic, to ensure comprehensive and nuanced reporting.
Develop
-
Identified and linked data points to outputs to quantify impact.
-
Added qualitative data to provide a holistic view and include case studies.
-
Recognised the need for iterative testing and evaluation of the framework longer term.
-
Piloted the framework with a sample report for a complex needs service:
-
Gathered data and consulted with service managers to interpret results.
-
Highlighted the necessity of integrating contextual knowledge for specificity.
-
Identified the need for a more efficient data aggregation system.
-
Recognized the time-consuming nature of producing impact reports across service areas.
-
Deliver
-
Using the framework, I created the first impact report for SJOG based on 2024 outputs and long-term outcomes for each service area, evidencing them with indicators to tell a compelling impact story.
-
Included stories of efforts to meet wider sector needs, the adaptation of services based on these needs, and the organisational response to policies and regulations.
Iterate
-
Learning from the first report, the framework is currently under iteration. This includes refining the indicators for each area a bit more and designing a format for colleagues to input these indicators easily, enabling faster creation of yearly impact reports.
-
A new set of information related to tenders and a repository to support fundraising efforts will be added.
Lessons Learnt
Making sense of existing data
A key lesson learned through this process was the importance of not overburdening staff with additional data collection efforts. To address this, I designed the framework to use existing data. This approach, despite some inconsistencies and variations on the data, allowed us to conduct a coherent analysis that aligned with SJOG's goals. This experience taught us that by leveraging the data already at hand and available resources, we could effectively demonstrate impact and allow the organisation to make informed decisions without putting extra strain on the team while we shape this process slowly.
Appointing the roles
One challenge I've encountered is that people are hesitant to engage with the tool because they don't feel they have the necessary expertise. Over time, I have learned to support my colleagues by showing them the importance of their roles and the value that using the framework brings to them. By involving them in designing the framework, collecting data, and analysing the findings, I have broken down barriers and demonstrated how this work showcases their efforts. My goal has been to create a framework that they can use independently, without relying solely on me to create a report or speak about their achievements.
Designing with nuances
An important aspect of this project is the complexity and diversity of the areas SJOG works in. It was crucial to strike a balance between having a streamlined, consistent method for reporting and not over-simplifying the needs of the people we support. Each story is highly personal, filled with traumatic and sensitive details, making it unfair to generalise outcomes and always quantify results. As a designer, I feel a responsibility to respect the experiences of the individuals supported and to honour the hard work of the project workers, while designing a tool that is useful for the organisation to visualise it’s impact ensuring consistent and evidence-based reporting that aligns with its needs and strategy.