Welcome to the Deep Web Library Guide. This guide will help you understand what the Deep Web is, why it is so difficult to search, and the efforts being made to uncover the information located within it.
The Deep Web consists of all information on the World Wide Web which cannot be located by general-purpose search engines (Google, Yahoo, Bing etc.). There's a large amount of information which even powerful engines such as these cannot find.
Sometimes you'll hear the term "Dark Web." These terms are sometimes used interchangeably, but the deep web and the dark web are not the same thing, as this helpful article from BrightPlanet explains: Clearing up Confusion: Deep Web vs. Dark Web
There is nothing inherently special about Deep Web pages. The Deep Web is largely the product of limitations in web crawler technology. As technology changes, so does the Deep Web.
A web page is often defined as a document on the World Wide Web with a single, unique web address. A web site is a group of web pages created by an individual, group or organization. Pages within a web site link to one another and are usually related by topic. One web site may contain many web pages.
Example: Saint Louis Community College has a web site. The opening page of the web site is http://www.stlcc.edu.
Within this site are numerous individual web pages, such as the STLCC Bookstores page:
Synonyms for Deep Web:
Synonyms for Web Crawler:
Open Web, Visible Web, Surface Web: The portion of the Web which is accessible to general search engines. This part of the Web is free of charge, available for use by anyone. However, note that there are free sites within the Deep Web as well, which for one reason or another are inaccessible to web crawlers.
Dark Web: part of the deep web, used for secure transactions and (often) criminal activity.
Estimating the size of the Deep Web is a difficult task, given the sheer number of web pages involved, and the hidden nature of those pages. However, BrightPlanet, a "deep web intelligence company" notes this:
"...it is important to tap into the rich resources existing in the Deep Web. The last time an extensive study was completed estimating the size of the Deep Web was in 2001 — a time when the internet consisted of only approximately three million different domains. The 2001 study revealed that at that time the Deep Web was approximately 400-500 times the size of the Surface Web.
Today’s internet is significantly bigger with an estimated 555 million domains, each containing thousands or millions of unique web pages. As the web continues to grow, so too will the Deep Web and the value attained from Deep Web content."
Source: Pederson, Steve. "Understanding the Deep Web in 10 Minutes." BrightPlanet, Mar. 2013. Web. 18 Sept. 2013.
|
St. Louis Community College Libraries |
Florissant Valley Campus Library |
Forest Park Campus Library |
Meramec Campus Library |
Wildwood Campus Library |
|
|