The Cart Shed impact management project

 
Presenting the new theory of change at the Cart Shed summer celebration.

Presenting the new theory of change at the Cart Shed summer celebration.

 
 

Case study: Data maturity

Client: The Cart Shed
Date: 2018-19
Work: Developing data maturity


INTRODUCTION

Data Orchard worked with The Cart Shed over one transformational year to focus, align, and mobilise the organisation towards greater impact.

The Cart Shed is a charity that enables people to manage their mental/physical health and well-being in a sustainable way. It operates outdoors in the natural environment and provides therapeutic support, creative and practical woodland/horticulture training, and group activities. It works with adults, young people and children including veterans and their families.

THE PROBLEM

The Cart Shed approached Data Orchard because they were:

  • Unclear about the impact they were having and lacked the knowledge and skills to measure it

  • Unable to extract any reliable or meaningful analysis for decision making despite investing huge amounts of staff and participant time capturing data

  • Frustrated with increasing numbers of unwieldy spreadsheets

  • Struggling with old and unreliable digital tools and systems

  • Unconfident about their data protection and security policies and practices.

Cart-Shed-word-cloud-BEFORE.png

At the outset of the project we asked all the staff to describe how they felt about data. This word cloud captures the essence of people’s attitudes. They knew it was important and useful but it was really DIFFICULT!

WHAT WE DID

We worked with The Cart Shed team (staff, trustees and volunteers) and through a series of workshops, meetings, and training sessions, we:

  • Reviewed the key areas of data capability (Uses, Data, Analysis, Leadership, Culture, Tools and Skills), identifying strengths, weaknesses and priorities.

  • Looked under the bonnet to assess digital tools and systems and produced a strategic plan for improvement.

  • Assessed staff roles, knowledge and skills.

  • Conducted a data audit.

  • Redefined the target population, inclusion/exclusion criteria, and referral processes for their beneficiaries.

  • Created a ‘Theory of Change’ and robust set of outcomes measures (short and long term) for evaluating impact.

  • Collated an evidence base of external research from government, academia, and local data relating to the needs and problems the charity addresses.

  • Implemented new digital tools and systems: standardised software, security, virus protection with charity licencing; migrated, cleaned and organised 30,000 files to a secure cloud-based shared storage solution; and advised on options and requirements for a new CRM (Customer Relationship Management) system.

  • Established new policies, practices and processes around the protection, security and management of beneficiary data.

  • Supported funding applications to secure financial resources.

  • Sourced suppliers and appropriate not-for-profit sector professional networks to connect them into good sources of knowledge and expertise.

Cart-Shed-montage-small.jpg

THE IMPACT

Data Orchard’s support has enabled The Cart Shed to establish clarity and consistency on how to 'tell their story' - about who they serve, what they do, and how they make a difference. Moreover, the project established the critical foundations (digital tools, processes, skills and knowledge) for them to continue building their data capabilities to both prove and improve their impact in the future.

Chart showing all staff responses (10 staff) using our data maturity assessment ‘before’ and ‘after’ support from Data Orchard.

Chart showing all staff responses (10 staff) using our data maturity assessment ‘before’ and ‘after’ support from Data Orchard.

Improved overall data maturity

At the outset of the project our data maturity assessment identified three priority areas: data, skills and tools.

Twelve months later the re-assessment showed there had been improvement across all seven themes of which skills was the most notable.

The right data

At the heart of the project, most importantly, was getting the right data.

The key steps we went through were to:

  • Co-produce a new theory of change and set of measures for capturing outcomes

  • Establish key questions for analysis to test the theory

  • Developed a set of key performance metrics with staff and trustees.

Cart-Shed-theory-of-change.jpg

Increased capacity, knowledge, and skills

Our initial review of staff roles and skills showed data was part of everyone’s job yet no one had any formal responsibility for it. Staff skills and knowledge in impact measurement was mixed and generally low. We ran a series of workshops and training sessions with the team and supported Katie, the CEO, to negotiate new responsibilities for existing staff e.g. managing digital tools, systems and licencing.

Clinical impact in the future will be overseen by the Clinical Director and measured and reported against the Theory of Change. Another manager took on responsibility for the implementation of a new CRM (Customer Relationship Management) system with support from a technically skilled volunteer. A new compliance officer was recruited to support consistency and accuracy in data collection, and to ensure all documents are filed and protected appropriately. In addition, new connections and relationships were established with external communities of practice and suppliers to ensure ongoing support and continued learning.

Boxplot chart showing how staff rated their knowledge and skills in impact measurement before and after Data Orchard’s support. The coloured box represents the middle range of responses, the ‘x’ is the mean average, and the horizontal line is the median average response.

Boxplot chart showing how staff rated their knowledge and skills in impact measurement before and after Data Orchard’s support. The coloured box represents the middle range of responses, the ‘x’ is the mean average, and the horizontal line is the median average response.

The chart above is interesting, as it shows an increase in the average (both mean and median), and that the range of responses has reduced. i.e. it’s not just one or two individuals who have increased their knowledge and skills, but there’s been an overall shift in the organisation as a whole.

Better digital tools

Our data maturity assessment showed tools were the charity’s weakest area.

A follow-up comprehensive review of digital tools and systems revealed:

  • much of the hardware was old and poor quality

  • software and licencing was patchy and inconsistent

  • data was inefficiently stored and difficult/impossible to share

  • there were many disparate systems for collecting and managing data.

Our recommendations set out a prioritised 3-5 year plan for building a robust, secure, reliable digital infrastructure.

Work to improve things began immediately:

  • New software, security and protection systems

  • Migration to a secure cloud file sharing platform

  • New digital devices for capturing data out in the woods

  • A new CRM system, plus plans for a wide range of other tools and systems to improve efficiency in information and communications.

Funding secured

Data Orchard’s independent assessment of needs and support with funding applications led to The Cart Shed securing £70,000 towards the cost of their impact management project, and improving their data, digital tools and systems.

Cultural change

Cart-Shed-word-cloud-AFTER.png

At the end of the project we asked staff again how they felt about data. The word cloud here, compiled from their comments, suggests ‘Useful’ and ‘Important’ are most prominent and whilst things are less ‘difficult’, data can still be seen as a necessary evil and a bit frustrating!

Perhaps most promising is a culture of ‘team approach’ to data. All the staff, trustees, and volunteers understand the organisation’s desired impact and why data is important. Almost all will be engaged in some way in the process of collecting, managing, or using data as part of their daily roles.

LOOKING FORWARD

The key foundations are now in place for The Cart Shed to continue their journey towards data maturity. It’s going to take time for new tools, systems, and impact measurement plans to become fully embedded before there’s enough reliable data for analysis.

There remain frustrating challenges with unreliable rural broadband and limitations of the CRM system (look out for our forthcoming provocation paper on the world of not-for-profit CRMS!)

The Cart Shed will spend the next 12 months:

  • Embedding new tools, in particular the new CRM system

  • Training and supporting staff to capture the right data in routine ways

  • Upgrading computers and other devices for collecting data

  • Ensuring these are properly supported.

Investing in long-term improvement will continue to take stamina, resources, and time. But they are able to clearly see the progress that’s been made already and have a road-map for the future.

 
 

I believe we will go from strength to strength. The knowledge and understanding about what impact is now flows through the entire team.

The experience has enabled us to have different kinds of conversations with commissioners (using the Theory of Change) with greater clarity about our purpose and a marked improvement in our overall professionalism.

We will continue to challenge and review, adapt, adopt and change as necessary. I hope we continue to grow and enable those we work with to live more wholeheartedly.
— Katie Eastaugh, CEO, The Cart Shed

 

FIND OUT MORE

You can follow the journey we travelled together in more detail by reading the four blog posts we published over the 12 months:

  1. I hate data

  2. Emerging from the data quagmire

  3. A theory of change is about owning your own story

  4. Becoming data savvy: how far can 12 months take you?

CAN WE HELP YOU TOO?