| "Best Practice" Evaluation Theory & Strategies
Helping clients make informed, data-based decisions
Theoretical Approach to Evaluation:
Quality Data in a Timely Manner
EEC employs the strategy of ongoing, formative evaluation as a key element in making data-based decisions and mid-course corrections, if warranted. To this end, EEC employs an evaluation approach that includes: process evaluation (quality of the content, design and delivery/implementation); outcome evaluation (value of outcomes); learnings (barriers/enablers, nuances, surprises, causal explanations/mechanisms/feedback loops); overarching questions about the value/worth of the program and forward/outward focused evaluation questions (e.g., replication, sustainability, threats, opportunities).
In order to benefit from this approach, it is essential to have high quality data that are available in a timely manner that address the interactions between and among stakeholders, stakeholder perceptions, the actual intervention, and the results. EEC ensures that only data that is "treasured" is measured.
Logic Model Development:
Identifying Project Goals & Outcomes
EEC evaluators value the development of a clear, sequential logic model that demonstrates the causal linkages from the onset of the project through goal attainment. The logic model illustrates the theory of change by linking the project objectives with the organizations and individuals working together to achieve them with the outputs of the project activities and, in turn, to the short-term, intermediate, and long-term outcomes associated with the project. EEC evaluators create draft logic models for client review and revision. When this review and revision process occurs at the beginning of a project, clients and evaluators start the work on the same page.
Evaluation Plan Development:
Indicators, Data Sources, Methods
Once the client and evaluators agree on the logic model or models, if there are multiple initiatives, EEC evaluators develop the evaluation plan. Typically, evaluation plans include overarching questions or performance indicators and outline data sources, methods and timelines for data collection, analysis and reporting. The client then reviews the plan for accuracy and coherence with the program goals, objectives and activities. The final plan serves as the roadmap for the evaluation and is reviewed at least annually and revised as needed.
Quantitative & Qualitative Mixed Approach
EEC uses a mixed-methodology approach to most evaluation projects, incorporating both quantitative and qualitative strategies to collect, analyze and report data. Typical evaluations include: development of surveys, interview and observation protocols, and focus group items. Fidelity of implementation data sets and end-user outcome data are analyzed and incorporated into formative and summative reports to gauge project progress, value and effectiveness.
All instruments and procedures are developed, tested, and implemented in accordance with standard evaluation protocols. Instruments are developed in a collaborative manner so that EEC evaluators utilize the content expertise of project staff. A range of surveys and instruments are designed based on the scope of the project.
EEC's philosophy is to conduct all program evaluations in a non-intrusive manner. Data collection strategies include face-to-face interviews, focus group and online/web-based methods. EEC evaluators will request access to student or client level data if warranted by the evaluation plan.
Tasks, Timeline and Management:
On Time & On Target
Once the evaluation plan is in final form, EEC staff develop a task/timeline chart to manage the evaluation. Typical task/timeline charts include scheduling of ongoing meetings with clients, instrument development, data collection, data analysis, interim reports to clients and summative reporting. This process serves to keep the projects on time and on target in accordance with the clients' and funding agency's expectations.
Reporting and Communication with Clients:
Ongoing Dialogue from Start to Finish
EEC is committed to an ongoing dialogue with clients. During project start-up, we expect several face-to-face meetings with the client to develop a solid working relationship. After the initial meetings, EEC has the capacity for participation in key management team meetings via video conferencing or phone. Interim and summative reports are tailored to address the stakeholder audience(s) and delivered to the agreed upon project timelines.