Evaluation Approach

EEC employs a systematic approach to evaluation that incorporates “best practice” evaluation theory and strategies. These elements are evident in all EEC projects and have been found to yield the data and information necessary for clients to make informed, data based decisions for programming purposes and to provide summative reports to key stakeholders.

EEC employs the strategy of ongoing, formative evaluation as a key element in making data based decisions and mid-course corrections, if warranted. To this end, EEC employs an evaluation approach that includes: process evaluation (quality of the content, design and delivery/implementation); outcome evaluation (value of outcomes); learnings (barriers/enablers, nuances, surprises, causal explanations/mechanisms/feedback loops); overarching questions about the value/worth of the program and forward/outward focused evaluation questions (e.g., replication, sustainability, threats, opportunities) (Davidson, 2009).

In order to benefit from this approach, it is essential to have high quality data that are available in a timely manner that address the interactions between and among stakeholders, stakeholder perceptions, the actual intervention, and the results. EEC ensures that only data that is “treasured” is measured.

EEC evaluators value the development of a clear, sequential logic model that demonstrates the causal linkages from the onset of the project through goal attainment. The logic model illustrates the theory of change by linking the project objectives with the organizations and individuals working together to achieve them with the outputs of the project activities and, in turn, to the short-term, intermediate, and long-term outcomes associated with the project. EEC evaluators create draft logic models for client review and revision.  When this review and revision process occurs at the beginning of a project, clients and evaluators start the work on the same page.

Once the client and evaluators agree on the logic model or models, if there are multiple initiatives, EEC evaluators develop the evaluation plan.  Typically, evaluation plans include overarching questions or performance indicators and  outline data sources, methods and timelines for data collection, analysis and reporting. The client then reviews the plan for accuracy and coherence with the program goals, objectives and activities. The final plan serves as the roadmap for the evaluation and is reviewed at least annually and revised as needed.

EEC uses a mixed-methodology approach to most evaluation projects, incorporating both quantitative and qualitative approaches to collect, analyze and report data. Typical evaluations include: development of surveys, interview and observation protocols, and focus group items to gather quantitative and qualitative data specific to the initiative in order to gauge project effectiveness.

All instruments and procedures are developed, tested, and implemented in accordance with standard evaluation protocols. Instruments are developed in a collaborative manner so that EEC evaluators utilize the content expertise of project staff. A range of surveys and instruments are designed based on the scope of the project.

EEC’s philosophy is to conduct all program evaluations in a non-intrusive manner. Data collection strategies include face-to-face interviews, focus group and online/web-based methods. EEC evaluators will request access to student or client level data if warranted by the evaluation plan.

Once the evaluation plan is in final form, EEC staff develop a task/timeline chart to manage the evaluation.  Typical task/timeline charts include scheduling of ongoing meetings with clients, instrument development, data collection, data analysis, interim reports to clients and summative reporting.  This process serves to keep the projects on time and on target in accordance with the clients’ and funding agency’s expectations.

EEC is committed to an ongoing dialogue with clients.  During project start-up, we expect several face-to-face meetings with the client to develop a solid working relationship.  After the initial meetings, EEC has the capacity for participation in key management team meetings via video conferencing or phone.  Interim and summative reports are tailored to address the stakeholder audience(s) and delivered to the agreed upon project timelines.