GETTING HELP

Building a Strong Foundation

Do we need outside help?

Sometimes you need outside help. While it’s important to build your internal skills and capacity for self-review, sometimes you need to call in the experts. That time may be when you feel unable to evaluate a programme.

Reasons to seek outside help may be:

For large scale or high stakes interventions.
When you need independence and an objective view.
For complex projects and situations.
When you need public accountability.
In sensitive or controversial circumstances.
When funding allows!

Tips for commissioning high-quality evaluation

Wehipeihana and Davidson suggest the following, to get a ‘real evaluation’:

Clearly define your purpose, uses for the conclusions and users.
Develop explicit questions to guide the evaluation.
Consider a broad range of options for evaluation approach and method.
Find out what skills the evaluation needs.
Commission the evaluation efficiently.
Actively manage the evaluation.
Review how the process went.

KAUPAPA MĀORI

Sometimes it can be useful to get someone from outside the organisation to help with evidence building. When is this necessary and what are the options?

EXPLORE

Further tips for commissioning evaluation:

Save time and energy with a two-phase process: “Expression of Interest” (EoIs) to shortlist qualified bidders followed by a full Request For Proposal (RFP) to select the right team.
Select contractors based on their capabilities, not an evaluation plan they may have written ‘blind’ (i.e. without speaking with stakeholders).
Consider not requesting a long final report at all – they are seldom read. Presentations and periodic, concise updates are more useful and usable.
Have a contingency plan for ‘pulling the plug’.

It’s crucial to be as clear as possible about what you want. Decide what work needs to be done.

That is, what decisions you must take, the depth of reporting on the evaluation, and when. Get the right ‘fit’ of evaluator/s. That means recruiting the right mix of skills, knowledge, experience and personality traits, as well as someone who fits into the team.

Be open and honest about the limits you’re working under. Explain the scale of the project and level of evidence required. That doesn’t mean that you need to reveal the budget but it saves a lot of time if you explain your resources and scale. Make sure your timeframes and budgets are realistic.

Consider external factors that could influence them both. You could ask a few providers for estimates before you pick one (see below).

A good evaluator will help you refine what you need and ask searching questions about what you need to get started. You may end up with something quite different from what you first thought. The evaluator may write the brief with you.

Be clear on the results you want but avoid being too rigid about a particular approach. The best commissioning and evaluation is negotiated when you trust each other. Leave room for a tenderer to be creative and make useful suggestions.

Be clear on how you want the results presented. Decide length, level of detail, format, use of visual methods and so on. You and the evaluator must both be clear what it is you anticipate they will produce.

There are a few ways to advertise for evaluation support:

WHAT	WHAT IT MEANS	WHAT'S GOOD AND BAD
Open tenders	Openly advertise the work.	Maximum range of tenderers, maximum competition, lowest strike rate, least likely to respond.
Select tenders	Invite a few parties to tender.	Higher strike rate, most likely to respond.
Expressions of Interest	Seek expressions of interest through an open or by invitation-only, then ask a select group to propose in a more detail.	More protracted but best of both worlds, efficient with everybody's time.
Word-of-mouth or by reputation	Ask around for good evaluators or contact people whose work you like.	Can save time and get the result you want but doesn't allow for others to express interest.

HOW TO DEVELOP A BAD PROPOSAL

Allow written questions but no direct contact.
Provide no indication of budget (or have no budget confirmed).
Withhold crucial documents (eg service description, organisation goals, evaluation framework).
Don’t provide information on programme data.
Project officer not available.
Overly onerous requests for proposal –this can exclude good people and not be a good use of everyone’s time.
Allow tenderers a short time to prepare proposals.
Require a detailed methodology and costing based on insufficient information.