How Search Engines find web-pages:
Search Engines follow links. They may be from anywhere on the web. The time it takes
between you publishing your page and the Search Engines getting to it may depend
on various factors.
Search Engines have software programs called bots or spiders that go around following
links from directories and other web-sites and pass that information back to the
Search Engine indexer.
Very often, once the Search Engine bot comes to any particular page on your site,
they try to get to the home page too. Hence it is very important to have a link
to your home page from every page on your web-site.
Once the bot reaches your home page it starts crawling through all the links on
that page. How many levels deep it crawls on the home page may depend on what Search
Engine is doing the crawling. Google is known to crawl all pages on your site; while
Yahoo is quite lazy about going through more than 2 or 3 clicks. If your page is
not accessible from your home-page via 3 clicks, it may not get indexed in Yahoo!
If for some reason, your page cannot be within 3 links from the home page, make
sure that there is a link going to such pages from outside.
Use of proper keywords:
You may already know that you need to target individual keyword phrases with properly
optimized content pages. What this also means is that you should not dilute the
effectiveness of any single page by having way too many different keyword phrases
on the same page. The best results can be obtained by having 3 to 5 primary search
terms on a page.
The first 10 words in your title tag are pretty important for page ranking; so use
your most important terms first. You could definitely avoid any modifiers unless
they are also part of your ranking search phrase.
Site Architecture and Sitemaps:
When your web-site grows, very often you tend to put web-pages in folders to make
organization easier. But that also means more clicks from the home page to get to
your web-page. This is also true when there are way too many links on your home-page
and you want to conserve PageRank for the most important pages(tier 2 pages).
A sitemap is very useful in this situation. All web-pages in your web-site should
be linked to from a Sitemap. This not only makes it easier for people to find your
pages, it ensures that any page on your site is within 2 clicks from your home-page.
You must know the difference between a sitemap that you use to submit your pages
to a search engine and the sitemap you employ for linking to your pages on the web-site.
The latter is merely a linking tool in regular html format.
Dead-end Pages:
Think of the spider going from page to page following links. Often it comes to a
page where there are no further links. If you are the user reading the page, you
have to click your browser's back button to go anywhere from that page.
Search Engines consider dead-end pages as bad for the web and their user base in
general. This is another reason why you should have at least a link to your home
page from every page. If you have other links pointing to other material within
your web-site or elsewhere, that would be considered even better.
Of course, if you have certain pages that you don't want the spiders to see, use
a Robots.txt
page and/or a noindex meta tag.
Summary:
- Have specific pages for specific search terms.
- Use links with
proper keywords for Anchor Text.
- Use a separate sitemap for human visitors.
- Avoid dead end pages by linking externally.