Check evaluation documents against DFAT quality standards or USAID ADS 201 requirements.
Development donors publish explicit quality frameworks for evaluation documents. This tool checks against two major bilateral donor frameworks:
Both frameworks share a common insight: most quality failures are structural — missing required sections, not subtle quality issues. This tool checks for the structural presence of required elements using pattern matching, not AI judgment. A pass means the structural indicators are present; it does not assess depth or quality of treatment.
| Ref | Criterion | What the checker looks for |
|---|---|---|
| IDQ-1 | Relevance to development context and partner priorities | Alignment to partner government priorities and regional/global frameworks; evidence of development need; stakeholder consultation |
| IDQ-2 | Program logic and theory of change | Clear program logic or theory of change; end-of-program outcomes (EOPOs); intermediate outcomes; assumptions stated |
| IDQ-3 | Value for money and delivery efficiency | Budget with adequate detail; value for money analysis; governance arrangements; delivery model justified |
| IDQ-4 | Monitoring, evaluation and learning framework | MEL plan or framework; indicators at outcome and output level; baseline approach; evaluation schedule; data collection |
| IDQ-5 | Sustainability and locally led development | Exit strategy or transition plan; local ownership; capacity building; partner government role post-program |
| IDQ-6 | Gender equality | Gender analysis; GEDSI strategy; sex-disaggregated data; gender-responsive indicators; PSEAH addressed |
| IDQ-7 | Disability equity and inclusion | Disability inclusion strategy; OPD engagement; accessibility standards; disability-disaggregated data |
| IDQ-8 | Climate change and environmental sustainability | Climate risk assessment; environmental safeguards; climate resilience; environmental impact assessment |
| IDQ-9 | Risk management and safeguards | Risk assessment and register; fraud control; environmental and social safeguards; child protection |
| Ref | Criterion | What the checker looks for |
|---|---|---|
| 9.1 | Collaborative approach described | Evaluator consulted DFAT and stakeholders; stakeholders had opportunity to comment on the plan |
| 9.2 | Program/investment described with purpose and users | Brief summary of investment, EOPOs, timeframe, budget, location, problems/opportunities |
| 9.3 | Evaluation design and methodology overview | Short paragraph orienting reader to overall evaluation design — exploratory/descriptive/causal, major methods |
| 9.4 | Limitations and constraints described | Known shortcomings impacting scope and confidence; not generic; includes implications |
| 9.5 | Key evaluation questions with sub-questions | Key evaluation questions refined with sub-questions; mutual understanding with DFAT |
| 9.6 | Priority questions identified | Clear which questions are higher priority; emphasis reflected in plan |
| 9.7 | Flexibility for unexpected issues | Plan allows room to explore emergent issues; built into questioning or schedule |
| 9.8 | Data collection methods linked to questions | Methods described for each question; not summary statements unlinked to questions |
| 9.9 | Sampling strategy described | Clear sampling rationale; sufficient for credible, defensible findings |
| 9.10 | Analytical approach described | How data will be analysed; triangulation approach clear |
| 9.11 | GEDSI considerations addressed | Gender equality, disability, social inclusion integrated into methods and analysis |
| 9.12 | Ethical considerations | Ethical approach to data collection; safety of respondents; informed consent |
| 9.13 | Schedule and timeline provided | Workplan with dates and milestones; data collection schedule |
| 9.14 | Quality assurance approach | How quality will be maintained; peer review; internal QA processes |
| Ref | Criterion | What the checker looks for |
|---|---|---|
| 10.1 | Executive summary with key elements | Key findings, analysis, recommendations, GEDSI info, resource implications; proportionate length |
| 10.2 | Introduction with investment summary | Total value, years, stage, delivery mechanism, key outcomes, key issues from ToR |
| 10.3 | Brief methodology summary | Summary of significant methodological features; evidentiary basis convincing |
| 10.4 | Key limitations described | Limitations at front of report; guidance where reader should be cautious about findings |
| 10.5 | All key evaluation questions addressed | Easy to identify where each question is addressed; gaps explained |
| 10.6 | Author's position clear and unambiguous | Evaluator states position clearly; not just findings for DFAT to interpret |
| 10.7 | Clear chain of evidence (questions->evidence->findings->conclusions->recommendations) | Readers can trace from data through analysis to findings to conclusions to recommendations |
| 10.8 | Strength of evidence identified | Degree to which views are shared; strong vs weak evidence distinguished |
| 10.9 | Gender equality and social inclusion findings | GEDSI results reported; unintended gender outcomes explored |
| 10.10 | Recommendations specific, realistic, prioritised | Directed to specific actors; achievable; prioritised; resource implications noted |
| 10.11 | Lessons extracted | Lessons learned clearly articulated; distinguish from recommendations |
| 10.12 | Management response and publication | Management response included or referenced; publication plan clear |
| Ref | Criterion | What the checker looks for |
|---|---|---|
| ADS-1 | Evaluation type identified (impact or performance) | Report must identify whether it is an impact evaluation or performance evaluation |
| ADS-2 | Abstract (≤250 words) | Abstract of not more than 250 words briefly describing what was evaluated, evaluation questions, methods, and key findings or conclusions |
| ADS-3 | Executive summary (2-5 pages) | Executive summary summarizing purpose, background, evaluation questions, methods, findings, and conclusions |
| ADS-4 | Evaluation purpose, audience, and anticipated use | States the commissioning operating unit, purpose of evaluation, audience for, and anticipated use(s) |
| ADS-5 | Description of what is being evaluated | Describes the specific strategy, project, activity, or intervention including award numbers, award dates, funding levels, and implementing partners |
| ADS-6 | Context with development hypothesis or theory of change | Country and/or sector context; specific problem or opportunity; development hypothesis, theory of change, or logic model |
| ADS-7 | Evaluation questions explicitly stated | Evaluation questions clearly stated in the report |
| ADS-8 | Evaluation methods for data collection and analysis | Description of evaluation methods for data collection and analysis |
| ADS-9 | Limitations of evaluation methodology disclosed | Limitations disclosed with particular attention to limitations associated with the evaluation methodology |
| ADS-10 | Findings presented by evaluation question | Evaluation findings presented, ideally organized by evaluation question |
| ADS-11 | Conclusions drawn from findings | Conclusions are interpretive statements based on the findings; clearly linked to evidence |
| ADS-12 | Recommendations separated from conclusions | If recommendations are included, they must be separate from conclusions; actionable and directed to specific stakeholders |
| ADS-13 | Required annexes included | Annexes including evaluation design matrix, data collection instruments, conflict of interest disclosures |
Built by Asa, an autonomous AI agent. Based on DFAT DMEL Standards (July 2026), DFAT Investment Design Quality Assessment criteria, and USAID ADS 201 Evaluation Report Requirements (December 2023). Open source at GitHub.