Our Approach

A systematic approach

Evaluation Theories & Strategy

EEC utilizes a systematic approach to evaluation that incorporates best practice evaluation theory and strategies. As outlined below, these elements yield the types of formative and summative data that clients appreciate. EEC works collaboratively with program leaders to assess their project’s purpose, design evaluations, and collect the necessary information for informed, data-based decisions.

Theoretical Approach to Evaluation

Most clients appreciate a holistic, comprehensive approach to evaluation that allows ongoing data collection, reflection, and utilization for service planning, midcourse corrections, and decision-making. Therefore, EEC approaches program implementation and evaluation using the principles of Implementation Science; we recognize that developing and measuring evidence-based practices requires thoughtful, systematic implementation processes in order for effective outcomes to be realized (Fixsen et al., 2005). EEC's approach to evaluation, therefore, starts with the client. Using a Participatory Evaluation approach, EEC seeks input from stakeholders at multiple levels to deeply understand program services and effects, identify relevant evaluation questions and process issues, and accurately interpret evaluation findings (Preskill & Catsambas, 2008). Other approaches include Developmental Evaluation and Case Study. Specific evaluation strategies include: process/formative evaluation (quality of the content, design and delivery/implementation); outcome/summative evaluation (value of outcomes); lessons learned (barriers/enablers, nuances, surprises, causal explanations); overarching questions about the value/worth of the program; and forward/outward focused evaluation questions (e.g., replication, sustainability, threats, opportunities) (Davidson, 2009).

Logic Model Development

EEC evaluators recognize the importance of logic models as critical tools that help programs identify their goals, plan their services, and specify the intended outcomes they aim to achieve using inputs and program processes. Essentially, a logic model is a visual presentation of how the project will work, illustrating the program's impact theory and assumptions (Donaldson, 2007), links between project objectives, individuals and organizations working to achieve them (Frechtling, 2007), external factors that may influence results, and the relationships among resources, activities and outcomes. EEC works with clients to develop detailed, conceptually complete logic models that provide the foundation for successfully moving a program forward according to its goals and objectives. We approach logic model development, review and refinement holistically, recognizing that the value of logic models is not only for program planning, but for implementation, evaluation and program performance-in essence, throughout the life of a program (Kellogg Foundation, 2004).

Evaluation Plan Development

Once the logic model is completed, EEC develops the evaluation plan. Typically, evaluation plans include overarching questions that guide the measurement and data collection strategies for assessing critical outputs, as well as short-term, intermediate and long-term outcomes. Evaluation plans also outline data sources, methods and timelines for collection, analysis and reporting. The client then reviews the plan for accuracy and coherence with program goals, objectives and activities. The final plan serves as the roadmap for the evaluation, is reviewed at least annually, and revised as needed.

Methodology

EEC uses a mixed-methodology approach to most evaluation projects, incorporating both quantitative and qualitative strategies to collect, analyze and report data. Typical evaluations include: development of surveys, individual and focus group interviews, observation protocols, and records/document review forms. Fidelity of implementation across sites or providers is also frequently measured. End-user data are analyzed and incorporated into formative and summative reports to gauge satisfaction as well as project progress, value and effectiveness. All instruments and procedures are developed, tested, and implemented in accordance with standard evaluation protocols (Fowler, 2008; Dillman, 2008; Krueger & Casey, 2000; Ruhe and Zumbo, 2009; Wholey et. al., 2010). Instruments are created in a collaborative manner allowing EEC evaluators to engage with project staff as well as benefit from their content expertise. EEC's strives to conduct all program evaluations in a non-intrusive manner, adapting as necessary to client needs and schedules. Data collection strategies include in-person and phone interviews (individual and focus group), online/web-based survey methods, and data extraction from program databases and documents. EEC evaluators engage in continuing education opportunities that allow them to stay abreast of current issues such as increasing survey response rates and data visualization.

Communication & Collaboration

EEC is committed to an ongoing dialogue with clients. During project start-up, we expect several face-to-face meetings with the client to develop a solid working relationship, establish communication channels and monthly evaluation work group calls, and identify relevant management and staff meetings in which EEC evaluators may participate Ongoing participation in key meetings provides valuable context for evaluation activities as well as the opportunity to inform evaluation methodologies, data collection strategies and data analysis. Furthermore, because EEC employs a cyclical evaluation approach that simultaneously plans for and reflects on data collections, these various meetings provide a platform in which EEC and project leaders and staff can review evaluation findings at strategic intervals, use formative and summative evaluation reports with key stakeholders to evaluate progress, and modify plans as appropriate. After initial meetings, EEC often participates via video conferencing or phone, attending in-person meetings at clients' request or for data collection.

Tasks, project management & reporting

Once the evaluation plan is finalized and agreed upon by project staff and funders, EEC develops a Data Collection Schedule to manage the evaluation. Typical schedules include timelines for meetings with clients, evaluation activities (e.g., instrument development, data collection, analysis), and interim and summative reporting. This process keeps projects on track, aligned with goals and projected outcomes, and in accordance with the client's and funding agency's expectations. Interim and summative reports are tailored to address the stakeholder audience(s) and delivered according to project timelines. Reports usually include quantitative and qualitative data that provide periodic performance feedback and examine the effectiveness of project activities as well as the achievement of outcomes. In addition to written reports, EEC also has the capacity to develop brief infographics as well as PowerPoint presentations. EEC evaluators are familiar with federal reporting requirements and assist clients in annual and final submissions of their Continuation Reports (e.g., 524B). Sample reports are available upon request.

Evaluation Ethics

EEC work is aligned with the American Evaluation Association (AEA) Guiding Principles for Evaluators and Program Evaluation Standards of Utility, Feasibility, Propriety and Accuracy. All EEC project personnel conduct their work in accordance with these principles and standards.

testimonials

Hear it from our clients

"The EEC team is very experienced, knowledgeable, organized, and enjoyable. They clearly identify evaluation questions and keep the work focused on these goals."
Project Co-Director
"I have appreciated the evaluation work that EEC has completed for our Center. The staff is very competent and professional to work with. I would highly recommend them to other grantees."
Center Director
"I am always pleased with the quality of work EEC does, as well as the collaborative spirit in which the firm undertakes project evaluation. EEC's staff strike a near-perfect balance between ensuring that funding agency requirements are met through evaluation reports and activities, while also actively working so that project evaluation can provide data that actively supports the completion and quality of project activities rather than just being about compliance and 'box-checking'."
Director of Federal Relations
The EEC team is thoughtful, responsive, and so helpful in brainstorming new ideas. They are a trusted partner in our work and I am so grateful to have them in our project. Plus, the people are wonderful!
Project Co-Director & Senior Internal Evaluator
"This is my first experience working directly with an evaluator. The consistency, organization, and positive approach is invaluable. "
Project Co-Director
"The team is very well coordinated and helpful. Great knowledge of federal grant requirements and interpretations of report content, needs, and preferences."
Project Co-Director
"The EEC team is very professional! I have confidence that they will keep the focus on meeting my project's evaluation goals so that we can stay on track with our project goals and timelines. The EEC team is flexible and understands when we need to make some adjustments in our data collection plans. They are professional and highly skilled in the work. I am grateful for the EEC team because they make working with them feel easy!"
Project Director
"The EEC team was very attentive at the proposal development stage, and then post-award as well. Their reputation in the field is well deserved, and it is clear why so many winning proposals have EEC as the evaluator."
Project Director
"EEC personnel are always easy to access and are very responsive. They support from conceptualization to analysis and everything in between. We've been able to incorporate written analyses into larger reporting efforts."
Technical Assistance Provider
"The work is always well done -- Pat and team help with conceptual thinking and then have really good follow through on collecting and reporting evaluation data. The reports are well done and thorough."
Project Director
"EEC collaborates with clients to produce responsive, quality work. I appreciate the dual focus on meeting reporting/funding requirements and generating actionable data."
Senior Internal Evaluator
"I would recommend EEC to colleagues for the following reasons: the consultative partnership approach that is part of the relationship; the trust that is developed; the working thought partner that we have who helps us reflect for continuous improvement and brainstorm for new approaches to challenges; the warm relationship that has been developed with our stakeholders; the guidance provided over time to help us keep our project on course as well as make course corrections along the way."
State Department Grant Director
"EEC staff is very knowledgeable about evaluation requirements that come from the funder, and the application of those to specific projects' goals and objectives. I very much appreciate their thought leadership and flexibility. Valued partners who improve our collective work."
Center Director
"EEC evaluation services are scientific, thorough, and well communicated. EEC staff work closely with the project team and ask insightful questions that influence the direction of the project, thereby the effectiveness of our work and our outcomes."
Center Director
"EEC sets the standard in program evaluation. Their methodologies are of the highest standard; they do an excellent job of translating data into accessible, useful reports; they meet every deadline; and they're a pleasure to work with. In addition, they remain current on regulations that impact their clients. When asked, they participate in funder/client meetings with complete professionalism and add significant value to those meetings."
Center Evaluation Team Lead
"Thoughtful and creative about methods and strategies to employ to assess the work; patient and persistent; produce high-quality work; organized and focused; helpful in brainstorming new ideas and researching approaches and strategies."
Center Evaluation Team Lead
"Strong expertise in the field of evaluation and educational assessment, close attention to activities and timelines, overall professionalism."
Center Evaluation Team Lead
"The EEC team is excellent – professional, easy to work with, timely, efficient and high quality. We feel confident that our evaluation work will be top notch when working with them."
Technical Assistance Provider
"EEC is collaborative, collegial and flexible while maintaining high standards for evaluating and documenting fidelity and quality of implementation. EEC's work has been critical in informing ongoing revisions to prototypes and our professional learning practices."
Technical Assistance Team Lead
"EEC leadership and staff have been extremely responsive to our project needs and bring deep expertise and experience with our OSEP client."
Center Director
"EEC employees have made a concerted effort to deeply understand my organization's mission and how we approach our work. They bring an unprecedented level of nuance to our evaluation efforts."
Center Co-Director
"EEC has considerable knowledge about evaluating projects that they are able to bring to bear. They also organize the evaluation process and provide leadership about next steps which is quite helpful to me as the Director. They also work collaboratively with the team. I am grateful that we asked them to be our evaluators."
Center Co-Director
"The EEC team is very knowledgeable about evaluation strategies, analysis procedures and always makes sound recommendations to clients."
Project Director
Previous
Next
Skip to content