Source: Devine, Jane, and Francine Egger-Sider. Going Beyond Google: The Invisible Web in Learning and Teaching. New York: Neal-Schuman, 2009. Print. Page 135.
The above diagram illustrates many of the concepts discussed in this guide.
Robots.txt is a file containing computer code which instructs crawlers how to crawl a given web page, or not to crawl it at all.
The Noindex Meta Tag is computer code appearing on a web page which prevents crawlers from indexing that page.
Relational databases are the kinds of databases already discussed in this guide: when searched, they provide results in dynamically generated pages. Note that Google's web crawler now searches inside many databases and indexes the results. These results are thus part of the Surface Web.
|
St. Louis Community College Libraries |
Florissant Valley Campus Library |
Forest Park Campus Library |
Meramec Campus Library |
Wildwood Campus Library |
|
|