The sitemaps protocol is a great and universal way to tell the search engines about all you're web pages - If (like me) you develop
alot of dynamic websites that are driven off large SQL databases,
trying to make sure all you're pages get crawled/found is really
important... This really easy sitemap is a must in you're web design toolkit.
In
this tutorial I am going to grab all the products from a database and
just loop through them (Using a SQLDataReader and then using the new
XmlTextWriter), you can use the same logic for pretty much every
website - I have used this script on about a dozen websites and just
changd the SQL to the appropriate table names and field names etc..
Firstly you want to make sure you have a robots.txt file in the root of you're website, and add the following to it
User-agent: *
Sitemap: http://www.YOUR-DOMAIN-NAME.co.uk/sitemap.aspx
Obviously
if you already have a robots.txt then just add the bolded line and
change the domain name to you're website domain - This tutorial will
create the sitemap.aspx file for you. More...