Build an agentic AI system that automates the entire meta-analysis workflow from research question to publication-ready results.
Meta-analyses represent the gold standard for evidence synthesis in medicine, yet conducting them requires months of manual work by teams of experts. We challenge participants to build an agentic AI system that automates this entire workflow.
Your system should accept a research question as input, such as "What is the effect of metformin on lifespan in animal models?", and autonomously produce a complete, publication-ready meta-analysis that matches the quality of expert human work.
The challenge will have a specific focus on biomedical interventions related to longevity and age-related diseases. This includes pharmaceuticals like metformin and rapamycin, as well as lifestyle interventions such as caloric restriction or the Mediterranean diet.
Precision and recall against gold-standard dataset curated by expert reviewers.
Agreement rates with manually extracted effect sizes, confidence intervals, and sample sizes.
Reproduce published meta-analyses within acceptable margins, correct heterogeneity handling.
Total computation time vs documented person-hours for manual meta-analysis.
Handle multiple study designs including randomized controlled trials, cohort studies, and case-control studies, as each requires different statistical approaches and quality assessment criteria.
Core task: automated classification of all selected articles across multiple domains. For each article, the system should identify:
Original research, systematic review, meta-analysis, case report, etc.
Blood biochemistry, RNA sequencing, DNA methylation, neurocognitive tests
Homo sapiens, Mus musculus, etc.
3-4 sentences: active agent, molecular targets, delivery method
Automated study selection process
Mortality & disease risk (aggregated data using coding agents)
Methods used to generate data (clinical trials, animal studies)
Confidence score based on study type and journal quality
Participants will receive three well-characterized research questions from different medical domains, each with existing high-quality published meta-analyses that serve as ground truth:
Pharmaceutical intervention with clear, standardized outcome measures (longevity intervention like metformin)
Behavioral intervention with more heterogeneous outcome reporting
Diagnostic test accuracy question requiring bivariate meta-analysis methods
Success in this challenge would fundamentally transform evidence-based medicine. Automated meta-analysis could reduce the typical timeline from months to hours, enabling real-time evidence synthesis as new studies emerge.
Systematic Reviews of Interventions
Reporting standards for your system
Borenstein et al. with code examples
Literature search and access
PDF data extraction
Statistical analysis implementation
Ready to revolutionize evidence-based medicine? Build the future of systematic reviews.