Quick question(s) about robots.txt

My robots.txt is in my site root, however my forum is in it's own folder. So my setup looks like this:

Code:

Disallow: /forums/archive/

My question is, does

Code:

Disallow: /forums/archive/

mean that anything starting in that URL will not be indexed? In terms of other folders, like images, would:

Code:

Disallow: /forums/images/

Disallow crawling of everything within the images folder?

And lastly, do the URLs and file extensions specified need to be the rewritten URLs, or the non-rewritten versions? Or do I need both?

Everything in robots is relative to the robots file itself, which must be in .com/robots.txt location (or it won't do anything).

so, yes, if you want to block /forums/images/ you need both folders to create the full path from root.

If you have non-rewriten items still indexed, yes you can block them, otherwise usually just the re-written version is fine.

Brian,

Might I ask what you or vbseo block or allow in your robots.txt files? My list has grown quite long. Should I be allowing tag pages for example? Or is it best to just block everything and only allow the sitemap to be crawled.

Also, would it be easier/involve less directives to just have:

Code:

Disallow: /forums/

To block everything within that directory, and then allow a few things selectively?:

Code:

Allow: /forums/tags/

You should ONLY block items that you don't want indexed. Blocking your forum directory is a bad idea.