- Meta searching: historical perspective (archie, veronica, etc), issues
- Current trends: commercial systems (Yahoo, Lycos, etc.) robots, STARTS, Lyc
- A. Paepcke, R. Brandriff, G. Janee, R. Larson, B. Ludaescher, S. Melnik,
S. Raghavan, "Search Middleware and the Simple Digital Library Interoperability
Protocol", D-Lib Magazine, 6(3), 2000.
SDLIP represents an important contribution to the DL interoperability
canon. It also clearly demonstrates some of the inherent difficulty
in constructing federation services.
- L. Gravano, C.-C. K. Chang, H. Garcia-Molina, A. Paepcke, "STARTS:
Stanford Proposal for Internet Meta-Searching," Proc. of the 1997 ACM
SIGMOD International Conference On Management of Data, 1997.
STARTS approaches metasearching by defining a interoperability protocol to
be implemented by the different search engines. STARTS defines the
mechanism by which proxies can query indices (of differing protocols) and
have enough standard meta-information to filter, rank, and display the
- C. Mic Bowman, Peter B. Danzig, Darren R. Hardy, Udi Manber, Michael F.
Schwartz, and Duane P. Wessels. "Harvest: A Scalable, Customizable
Discovery and Access System. Technical Report CU-CS-732-94, Department of
Computer Science, University of Colorado, Boulder, August 1994 (revised
If you don't want to do protocol conversion for different indices,
or you cannot rely on the indices to comply with a protocol such as
STARTS, then for some applications it is reasonable to gather the remote
information yourself. The architecture of most commercial systems
(Altavista, Lycos, etc.) is proprietary information, but Harvest is a
freely available and popular system for gathering and serving remote
information that incorporates all the general components of its commercial
brethren. It has a clean, modular design and has the ability to
hierarchically arrange different Harvest servers.