The anatomy of a Large-Scale Hypertextual Web Search Engine

What we want from a search engine.

•Speed•Quantity of Results•Efficient Storage Space•Quality of Results

Google attempts to bring us all of these aspects from search.

Precision of result:

Page RankAnchor Text

Second Generation Search Engine

Page Rank

PR(A) = (1-d) + d(PR(T1)/C(T1) + … + PR(Tn)/C(Tn))

The more number of links that is pointing to a page (from other pages), the higher the page rank will be.The probability that a random internet surfer will reach this page by randomly clicking links.

Also determined by the number of links the page has pointing you have.The more links page A has, the more valued the link from page A to B will be.

Anchor Text

Each and every link on the internet will have some “invisible” text alongside it. This text is given by the page creator explaining what this link does, where it leads, or what it attempts to explain. By taking all of these links from hundreds of different sites, Google uses these anchor text to be able to provide most relevant search results.

Proximity Search and Others

Google keeps track of how close the related words are too each other and also keeps track of the visual presentation (font size, color, boldness ect).

Crawling and Indexing

•Google typically ran about 3.•Each crawler opens roughly 300 connections as once.•At peak performance, with 4 crawlers, Google can crawl 100 web pages per second.•Roughly 600K per second of data.

•Parsing•Indexing documents into barrels•Sorting