Zillusion · discovery
It finds the source. You keep the wheel.
The right API, file, or table is rarely the first search result — often it’s an undocumented portal API that takes days to find by hand. Zillusion’s discovery stage turns “find me this data” into a ranked list of every public path to it.
Why discovery is the expensive part
Most of the cost of web data is discovery and maintenance, not analysis. Before anyone writes a line of extraction code, someone has to answer: does an official API exist? Is there a bulk file? A government portal? An undocumented JSON endpoint behind the page? Teams usually answer this with days of manual research, and often settle for scraping a rendered page when a stable API was one request away.
How Zillusion discovers sources
From one plain-language sentence, the discovery stage fans out across:
- 4 web search engines, queried in parallel rather than relying on a single index;
- vertical registries — the portals and directories where structured data actually lives;
- a curated API catalog with 23,000+ rows, retrieved semantically in both English and Chinese using dense embeddings plus BM25, fused with reciprocal rank fusion.
Every candidate is scored on 5 dimensions and grouped by access type — APIs, downloadable files, or data embedded in pages — so the trade-offs are visible before anything is built. The build stage then prefers stable APIs and files over brittle HTML selectors whenever a ranked candidate offers one.
A concrete example
In the recorded run replayed on our homepage, the request was “Global CO₂ emissions by country — CSV or JSON.” Discovery returned 13 candidate sources, grouped and scored; the top hit was the Government of Canada’s Greenhouse Gas Emissions indicator (CESI) at a score of 8.6. The agent then probed the access paths — API? CSV? page table? — before writing the workflow. The result: 63 records in 3 CSV files, verified against the live source, gate-computed PASS, in 9.5 seconds of clean runtime.
Ranked, not decided for you
Discovery ends in a ranked, grouped list — not a silent choice. You can accept the top candidate, pick another path, or point the agent somewhere it hasn’t looked. Most tools in this space need you to supply the URL; Zillusion’s bet is that finding and ranking every public path is the half of the job that deserved automation first.
Frequently asked questions
How does Zillusion find data sources from a plain-language request?
It fans out across 4 web search engines, vertical registries, and a curated 23,000-row API catalog with bilingual (English and Chinese) semantic retrieval — dense embeddings plus BM25, combined with reciprocal rank fusion. Candidates are scored on 5 dimensions and grouped as APIs, files, or embedded data.
Why does it prefer APIs over scraping the page?
APIs and bulk files are stable contracts; HTML selectors silently rot when a site redesigns. Zillusion's build stage probes the ranked access paths and prefers an API or file whenever one exists, falling back to page extraction only when that is the best public path.
Can I override what discovery picks?
Yes. Discovery produces a ranked, grouped list of candidate sources, and the run stays steerable: you can pick a different candidate or direct the agent to a source you already know before anything is built.
Does discovery work in languages other than English?
The API-catalog retrieval is bilingual (English and Chinese) by design, and the search fan-out follows the language of your request.