After a Search Engine spider or robot has found, downloaded and stored the page another program, called an indexer, processes the page using its own proprietary algorithm.
A search engine indexer will carve the page up into its various components removing all the HTML tags and store links in a queue. The contents of the meta keyword tag, description tag, titles, headings, links, body copy, bold, and italic are analysed. Some indexers compress the page by removing stop words; finally the much cut down page is stored in an online searchable database.
|