Management for Evaluators and Those with Oversight Responsibility

Description: Evaluation theory and methods meet the reality of program operations and decision-making when a program evaluation is implemented. Against a backdrop of demanding technical requirements and a dynamic political environment, the goal of evaluation management is to develop, with available resources and time, valid and useful measurement information. Management responsibility rests with a project director and, to a lesser extent, other project staff. Those who oversee evaluations, usually as sponsors' representatives, also have management responsibilities, such as developing specifications for a call for proposals, and selecting and monitoring the evaluat ion team . Evaluation management is challenging because the choice of methods (quantitative or qualitative) and the prevailing political context varies across projects.

The purpose of this course is to increase the evaluation management knowledge and skills of those who conduct evaluations (as project directors or staff) and others who have oversight responsibility (for programs and studies). Participants will have opportunities to improve their ability to directly manage or oversee each phase of an evaluation. Using a combination of short presentations, seminar discussions, and small group exercises, the course will focus on six areas: (1) selecting the evaluator(s); (2) developing rational work plans; (3) organizing staff for results; (4) making assignments productive; (5) monitoring interim progress; and (6) ensuring product quality and usefulness. Practical suggestions will focus on how common problems can be avoided or resolved. In addition, cross-cutting topics will be addressed such as: how the six areas of evaluation management are interrelated; how project management determines the quality and usefulness of evaluation products; and, why maintaining agreement with sponsors on the evaluation mandate is crucial. Case examples of evaluation management challenges and solutions will be examined and used in class exercises. The case examples will be provided by the instructor as well as drawn from course participants' evaluation experiences.

Instructor: James Bell is the president of James Bell Associates, Inc., a 35-person firm that has specialized in national health and human services program evaluation for 27 years. He has 32 years of evaluation management experience on more than 100 projects sponsored by federal, state, and local government agencies and non-profit foundations. This course will draw from Chapter 20. Managing Evaluation Projects, which he wrote for the Handbook of Practical Program Evaluation: Second Edition (Wholey, Hatry, Newcomer editors, Jossey-Bass, 2004) as well as earlier publications including Evaluation and the Federal Decision Maker (New Directions for Program Evaluation, Jossey-Bass, 1990). His project management experience spans an array of evaluation designs and programmatic areas, including: exploratory case studies of innovations in rural health care finance and delivery; design of a random assignment evaluation of a promising foster care prevention intervention; and, a nationally representative survey on protections for human research subjects. Recently, he evaluated models of integrated care for persons living with HIV and co-morbid psychiatric and addiction disorders. He led a ground-breaking $40 million multi-site cooperative research program in this area that was jointly funded by the National Institutes of Health and two other federal agencies. From 1974 to 1979, he worked with Joseph Wholey and other members of the Urban Institute's Program Evaluation Studies Group to develop logic models, evaluability assessment and other approaches to planning useful program evaluations.

Certificates: CEP IA.f

Fee: $795