What is going wrong?

After installing vBSEO my search engines results have been effected for the worse. My forum is AdminFuel I don't know if this is a temp. drop and things will pick up, but still.

1) I have attached an image showing how much Googlebot crawls my forum. As you can see, there is a sudden drop in the crawling after vbSEO was installed. What is the problem with things?

2) My search engine results have dropped from 6000 to 400, i am not sure why. My robots.txt is as follows, and has been built from information here:

Code:

# Allow Archiver
User-agent: ia_archiver
Allow: /


User-agent: Slurp
Crawl-delay: 60


User-agent: *

Disallow: *.php
Disallow: *.js
Disallow: *.jsp
Disallow: *.cfm
Disallow: *.asp
Disallow: *.aspx
Disallow: *.cgi

Disallow: /includes/
Disallow: /install/
Disallow: /customavatars/
Disallow: /archive/
Disallow: /sitemap/

Sitemap: http://www.adminfuel.com/sitemap_index.xml.gz

Is there something wrong? I have URLS that end in .html so there shouldn't be a problem right?

3) I have set up my sitemap with the modification you profide, but the number of URLs it is picking up is only around 200, and this has dropped from 2000+

What am I doing wrong?

Can you pleas provide me with good feedback.

You should not be blocking *.php in your robots.txt file. You need to allow the search engines to follow those old php files so that they can hit the 301s and release the new URL formats.

If you have more than 200 thread pages (public) then your sitemap should be including your total public threads+, certainly not less.

Please fix your robots.txt file, and open a support ticket to have a member of the Total Support Team check your sitemap settings. Please include username + password for your vB AdminCP.
vBSEO.com Helpdesk