Here you'll find the process for conducting an impact evaluation. Although the process for conducting an impact evaluation is laid out as a series of sequential steps, it should be emphasized that doing so does not mean that the process is either strictly linear or formulaic.

Conducting a successful study requires a federal evaluator (and hired outside evaluator, if procured) with experience in both the art and science of evaluation, able to capture the complex and multiple direct and indirect effects of public investment using creativity and a variety of implementation approaches. The process must be viewed as iterative and dynamic.

The seven steps below appear in the order in which they are often performed in practice. The detailed bullets can also serve as a checklist for executing an impact evaluation process.

However, as with all processes of research and inquiry, most of the steps are iterative in execution and involve feedback loops. The steps are not prescriptive, but they do represent common practice for evaluations.

The EERE Guide for Managing Program Evaluations and EERE R&D Evaluation Method Guide provide details on how to conduct impact evaluations.

Phase 1: Plan Evaluation

Step 1. Initial Preparation for the Evaluation

(Federal staff role – Evaluation Manager)

  • Identify program areas for study.
  • Consider what decisions need to be made and the timing for the decisions.
  • Formulate prioritized questions that if answered can provide evidence needed to inform decision making, formulating an initial program logic model will help.
  • Decide on the evaluation objectives and consider the type of evaluation(s) needed to develop the information to answer the questions.
  • Determine the funds needed to support the evaluation.
  • Determine a timeline for completing the evaluation and align timeline with when information is needed.
  • Determine the level of evaluation rigor needed to satisfy the intended uses of the results.
  • Begin organizing program background data and records for use in the evaluation.

Step 2. Hire an Independent Outside Professional Evaluator

(Federal staff role – Evaluation Manager, Contracting Officer)

  • Develop the Statement of Work (SOW).
  • Develop a Request for Proposal (RFP) for use in select an Evaluation Contractor.
  • Implement the RFP competitive solicitation process to hire an independent evaluator.
  • Ensure a process for external peer review of the study is specified in the RFP and SOW.

Step 3. Develop the Evaluation Plan

(Evaluator role)

  • Develop a final program logic model; the hired evaluator will construct or confirm the final program logic model.
  • Finalize the researchable questions the evaluation must answer.
  • Define the evaluation outcome and impact metrics.
  • Perform an evaluability assessment.
  • Determine and develop the type(s) of research design needed to answer the evaluation Questions.
  • Identify data to be collected and develop a data collection plan.
  • Select the analytical methods for answering the evaluation questions. 
  • Choose the appropriate analytical method(s) for the selected research design.
  • Prepare a technical Evaluation Plan and participate in the peer review process established for the plan.

Phase 2: Conduct Evaluation

Step 4. Perform the Evaluation

(Evaluator role)

  • Perform sampling, data collection, and measurement and verification. 
  • Complete data analyses and calculations, including sensitivity analysis as necessary.
  • Identify key findings. 
  • Prepare draft and final report.
  • Participate in the peer review process established for the draft report.

Step 5. Monitor the Evaluation Project During Implementation

(Federal staff role – Evaluation Manager)

  • Hold and participate in periodic project progress-review meetings.
  • Review project status reports submitted by the hired third party evaluator.
  • Monitor evaluator’s achievement of milestones and expenditures.
  • Establish and manage the internal and external review process created for the study. 
  • Anticipate and address management challenges during the study, and ensure technical challenges are addressed by the hired Evaluator.

Phase 3: Produce, Disseminate, & Use Results

Step 6. Report the Evaluation Results

(Federal staff role – Evaluation Manager)

  • Participate in peer review of draft evaluation report. 
  • Review the final report.
  • Publish final report.

Step 7. Use the Evaluation Findings

(Federal staff role – Evaluation Manager)

  • Implement a distribution strategy for the final report and results. 
  • Use the results to make decisions about the program.
  • Make, or monitor the making of, decisions about the program Based on the evaluation results.

 

Related