G-stockstudio / Shutterstock

The Why and How of Data-Driven Impact Management

November 30, 2017

According to New Philanthropy Capital‘s 2017 report on Global Innovation in Measurement and Evaluation Report, “impact management involves integrating impact assessment into strategy and performance management by regularly analyzing and responding to data and using it to change and improve.” Adopting impact management makes an organization agile through frequently collecting data and rapidly responding to it in real time. To embrace Impact Management, organizations can leverage technology to make data collection and data sharing seamless.

There are several limitations of conducting a traditional Impact Evaluation for a program.

In a traditional impact evaluation, data collection is often a one-time activity, done after-the-fact, to answer a limited set of questions. This after-the-fact data collection means resources have already been spent, without the opportunity to correct to a more effective intervention. This is not necessarily the best use of limited resources, and puts a burden of data collection onto field-workers and the program's target population without giving them anything in return.

Impact management shifts the focus to frequent data collection and iterative course correction. The hallmarks of impact management are collecting data as a matter of course during program activities, analyzing it regularly, and adapting the program in response to the data.

A shift to more frequent data collection, though, is often met with resistance within an organization. This resistance can't just be attributed to “organizational inertia”. Part of it stems from the traditional flow of data across different levels of organizational hierarchy. Another part stems from frustration with cumbersome data collection processes. A combination of open leadership and technology solutions can address both the problems.

Let's look at the typical flow of data in an organization. The leadership tends to have access to organization-wide data that was collected by field-workers and aggregated by administrators. But this flow is unidirectional and the loop isn't closed by sharing the data back within the organization. Hence, as a leader, you'd like more data because that enables you to make better informed decisions. However, if you're an administrator managing workers that would be collecting this data, you may be groaning inwardly at the thought of resources spent on data collation and reporting. I have met many field workers who think that documentation and data collection are a distraction from 'real' work.

Creating personalized feedback for everyone to close the information loop would be inefficient, but democratizing data can work around this problem. For example, creating an organization-wide dashboard that tracks program-specific performance indicators will enable everyone to assess their own performance. This in turn will help to re-frame the conversation around data collection because it allows everyone to understand their work in relation to program outcomes and organization's larger vision.

'Monitoring', with its Big Brother undertones, may tempt personnel to inflate their performance. Shifting the conversation to management, with an emphasis on responding to emerging situations, gives everyone a sense of ownership and promotes collective problem-solving. The shift changes data collection from an event to a matter of course, and data review from siloes to an open process.

Next let's look at the data collection process. In a traditional impact evaluation, organizations have no incentive to develop seamless processes for a single data-drop. Again, ubiquitous technology like smartphones can be used to automate data collection. This would allow organizations to collect richer data in the form of images, geographic coordinates, audio, video, etc. without adding significantly to field workers’ workloads.

In conclusion, impact management makes the best use of an organization's resources. At its core, impact management is fine-tuning an intervention by responding to feedback. Getting feedback means more frequent data collection. Responding to the feedback means analyzing data to identify specific changes to the intervention. To implement impact management, the organization must leverage modern communications technology to collect data seamlessly and to share collected data. Sharing data allows field workers to understand their own work in the context of broader organization goals. Consequently, they are able to identify specific problems in the data and possible solutions. Switching to a painless process and sharing data helps field workers to see the value of data collection, and do it more carefully.

Contact Information

Fels Institute of Government
University of Pennsylvania
3814 Walnut Street
Philadelphia, PA 19104

Phone: (215) 898-2600
Fax: (215) 746-2829

felsinstitute@sas.upenn.edu