Captain (YC F25) is building an AI-native indexing and querying platform for large-scale unstructured data. Alongside ML@Purdue and Purdue Ventures Consulting, we’re building DeepQuery.org — an open-source, unbiased benchmarking platform for enterprise AI search tools.
DeepQuery.org is an interactive, research-grade platform for evaluating enterprise retrieval systems, inspired by tools like LMArena.
Goals
- Implement a benchmarking pipeline supporting multiple retrieval architectures (Captain, RAG, GraphRAG, Elasticsearch, and others)
- Host small, medium, and large datasets under a tier system to measure speed, accuracy, and cost
- Display results in a public arena-style interface where users compare tool outputs side by side
- Publish a research paper covering methodology, results, and use-case recommendations
This work serves the broader AI community by clarifying when to use different retrieval approaches, reducing vendor hype, and guiding enterprise adoption.
Project members gain industry experience, research experience, inside access to Captain, and a close view of startup life.
Getting familiar with Captain
- First Look into Captain — AWS BuildersLoft (2-minute lightning talk)
- captain.dev