Multi-agent hospital workflow benchmark
Digital Hospital Environment
An open-source clinical AI benchmark environment for agents operating inside structured hospital workflows. It combines role-specific knowledge checks, patient-facing clinical operations, cross-role communication, deterministic grading, dense process rewards, and rollout capture.
11hospital agents
47patient cases
47hidden answer keys
550role-specific MCQs
2,200MCQ answer options
55unique tools
7,436non-comment Python lines
11,280custom logic surface elements
Environment design and evaluation
- Digital Hospital evaluates process, not only final medical answers. The agent must triage, inspect charts, use diagnostic tools, recognize critical findings, communicate when appropriate, and submit evidence-supported treatment plans.
- Roles include emergency physician, intensivist, cardiologist, surgeon, nephrologist, senior hospitalist, neurologist, infectious disease physician, clinical pharmacist, clinical research expert, and hospital director.
- The package includes patient records, hidden answer keys, role-specific MCQ banks, FastAPI runtime, tool handlers, graders, communication routes, trajectory collection scripts, Docker configuration, and inference runners.
- The environment supports model evaluation, process-supervision datasets, offline RL experiments, multi-agent workflow research, director-level feedback, and reproducible failure analysis through deterministic hidden grading logic.
Best suited for
- Multi-agent clinical workflow evaluation
- Trajectory collection for SFT and preference learning
- Offline policy improvement
- Clinical tool-use and evidence-before-treatment discipline testing
