What Makes an Effective Policy Evaluation?

You are here

Max Griboedov / Shutterstock
December 8, 2017
Hanzhe Xue

Think tanks have a major role in policy advocacy and policy evaluation, but, how do they provide correct and practical policy suggestions to local and central governments? Whilst working at the Chinese Academy of Macroeconomic Research, a think tank in Beijing, China, I thought about this question. In my two projects, completing a feasibility analysis for a proposed policy and evaluating a land management policy in progress, I found three elements which are essential for effective policy evaluation.

Think in a macro scope and promote interdisciplinary collaboration. Quite different from working in a business setting, which usually focuses on one problem at one time, policies have a wide influence and can address various problems at one time. Take land management policy, which entitles the farmers of the land’s ownership and management right and therefore encourages more flexible land usage: with land management right validation, land owners could now rent out their lands to firms instead of cultivating by themselves, which would boost economic transformation in suburban area. Further by validating the land to widow/er of the original landowner, the policy could also serve an important role in gender equality and women's empowerment. Since this single policy could generate a series of effects on different areas, it’s important for people who work in the public sector to think in a macro scope and consider possible outcomes comprehensively. This should be supplemented with work input from experts from different fields, promoting interdisciplinary collaboration. These two elements can give us a well-rounded understanding of the policy, and therefore an effective evaluation.

Down to earth. Desk research is never the full story of think tanks. Although the daily work in a think tank involves a lot of research, what makes policy research practical and valuable is field work. The vast amount of information accessible online is a useful resource, but many data sets don’t reveal the color and details; further, reliance news can lead to biased or exaggerated analysis, which cannot serve as valid evidence. Therefore, field research and interviews can be quite useful. Firsthand materials obtained through in person interviews with farmers and local officials involved in land management indicated the gap between the ideal and reality. This allowed us to give policy makers practical and specific suggestions based on the real condition of land management today.

Measureable Goals. As with program evaluation, SMART principles (Specific, Measurable, Achievable, Relevant and Time bound) are also applicable to policy evaluation. For evaluation, quantified and measurable indicators would be helpful to determine whether the program has achieved its goals. In the land management program for example, the ultimate goal is to validate all the land in suburban areas of China, while some county-level governments didn’t bond the goal with certain time frame and thus lagging behind, it’s hard to evaluate the overall progress and effect. Therefore, measurable goals will be highly effective for both execution and evaluation.

There is a Chinese saying, “Touch the sky, down to earth”. This is particularly fit for the work of think tanks. As these organizations are working to provide policy suggestions for government, they require the essence of macro scope combined with detailed plans, all of which should be developed and optimized through continuous research and abundant field work to result in a reliable and practical guidance with pertinence that can benefit the targeted population.

Fels Institute of Government

The Fels Institute of Government
3814 Walnut St.
Philadelphia, PA 19104

(215) 898-7326
felsinstitute@sas.upenn.edu

Facebook   Twitter   YouTube