URL's not followed... possible errors in redirect..

It's a good idea to add them, but not necessary. As long as you can see they are being downloaded, that's what's important.

I don't think the Google Group is gonna be that helpful for us on figuring out why I have those URL's not followed.

I just checked... my last sitemap continues to show over 4900 errors. Here are a few samples that were not there the other day:

HDTV finally comes to head|HDBeat|Home Theater Shack Redirect error [?] Sep 24, 2006
http://www.hometheatershack.com/foru...op-box-announc... Redirect error [?] Sep 24, 2006
http://www.hometheatershack.com/foru...ead-rising-pla... Redirect error [?] Sep 24, 2006

Those URLs aren't cut off in your sitemap files are they? I assume they just show a partial URL there, since otherwise it would probably be under the 404 section.

FWIW, I checked my Google Sitemaps on the two sites I currently have set up, and have 0 of these errors on either. Strange.

I'm not sure why it cut those off when I copied to here... here's a few more:

Thomson's Film Grain Technology Specification Approved by DVD Forum|Home Theater Blogs|Home Theater Shack Redirect error [?] Sep 22, 2006

WTB: BFD 1124 and/or Calibrated Mic|Home Theater Classifieds|Home Theater Shack Redirect error [?] Sep 24, 2006


Okay... yeah, I see what's happening. It list only part of some URL's but if you click on the link it reflects the full URL... when I copy, it only grabs what's showing on the screen.

You not have any errors on your site makes be think it's not a Google Webmaster issue, but something to do with how I have vBSEO setup or my .htaccess setup. It has to do with redirect errors and the only redirects I have are a result of vBSEO, except one in my .htaccess file in my public_html directory... which should not effect my /forums/ directory.

Still no valid help from Google Group other than they wanted to know if I tested my .htaccess. I posted if for them because I have no way of testing it.

Checked the tools and we are now up to 5155 URL's not followed due to rewrite error... here's a few to show the latest date:

The SWAMP! - Volume One|Chat Box|Page 10|Home Theater Shack Redirect error [?] Sep 25, 2006
The SWAMP! - Volume One|Chat Box|Page 11|Home Theater Shack Redirect error [?] Sep 26, 2006
The SWAMP! - Volume One|Chat Box|Page 12|Home Theater Shack Redirect error [?] Sep 26, 2006

Got a response from the Google Group... but I can't answer it...

My first guess is that the URL redirects too many times. We suggest fewer than 5 total redirects:
Webmaster Help Center - 3xx (Redirected)

How many times does the page redirect?

Of course the count keeps growing:

URLs not followed (5492)

Hello Sonnie,

there is only one-step redirect from these URL (this can be checked with any http headers viewer application), so this must be related to something else. I couldn't imagine what might be wrong here

Is my .htaccess the same as everyone elses? Can someone look at my setting and see if there is something that looks whacky?

This is a lot of URL's that are not being listed. I feel like I would do better without vBSEO, but I hate wasting money on a product and it not work for me.

It's a rewrite error and vBSEO is the only thing that I have setup to do rewrites.

Is my .htaccess the same as everyone elses?

The redirect command can be found in http headers (it's not required to have .htaccess or any other code of the site to check this) and it is correct in your case.

The SWAMP! - Volume One|Chat Box|Page 10|Home Theater Shack Redirect error [?] Sep 25, 2006
The SWAMP! - Volume One|Chat Box|Page 11|Home Theater Shack Redirect error [?] Sep 26, 2006
The SWAMP! - Volume One|Chat Box|Page 12|Home Theater Shack Redirect error [?] Sep 26, 2006

Notice that the redirects showing the problems are those from your original archive location: archive/index.php/this_file.html - These are native redirects vBSEO does to consolidate the old archive URLs to the new core location for the threads. I just tested and they are '301-redirected' in one step. These are non-existent files.

Have you gotten errors from redirects other than files from archive/index.php?

Yes... I believe there are more than just archives with errors, if this is what you are asking. Here is another recent one:

Acoustic Treatment Project|Home Theater Acoustics|Home Theater Shack Redirect error [?] Sep 27, 2006

Here is what someone in the Google Group posted, but I don't really follow it or know how to respond or what to do:

Can you have a look at your server access logs?
Could it be something to do with cookies set by some of these URLs?
The redirect is complex, could be some conflict there with the
robots.txt rules?
Also, as an example, the URL
The SWAMP! - Volume One|Chat Box|Page 10|Home Theater Shack
redirects to a page of more than 200,000 bytes,
quite large for an HTML document,
maybe bots do not have time to wait until they get such a large page?
It is all quite mysterious.

Another response from Google Groups...

There are a few very strange rule's in there. Are you sure that some
of that is not supposed to be commented out?

as
RewriteCond %{REQUEST_FILENAME} -f
RewriteCond %{REQUEST_FILENAME}
(admincp|modcp|clientscript|cpstyles|images)/

RewriteRule ^(.*)$ $1 [L]
doesnt seem to have a purpose except use apache resorces.

RewriteCond%{REQUEST_FILENAME} -f will match any file so includes the
rule on the next line
RewriteRule ^(.*)$ $1 [L] will rewrite everything to itself and is the
last rule.

below is the link to the apache site, so you can have a look at trying
to sort out the rules.
http://httpd.apache.org/docs/1.3/mod/mod_rewrite.html

Just ask your host were your server logs are, they are probabley in
your root directory in a directory called logs or stats, something like
that.

example -
66.249.65.71 - - [01/Oct/2006:05:20:18 +0100] "GET
/download/flash/course_booking_form.swf HTTP/1.1" 200 104414 "-"
"Mozilla/5.0 (compatible; Googlebot/2.1;
+http://www.google.com/bot.html)" "-;-"

IP address - - date "request_type file_asked_for HTTP version"
response_code size user_agent
You are looking for anyting that is showing a 301 response as that is
something that your htaccess is causing with its first rule. None of
the other rules are sending a redirect response, so wont show up in
your logs.
Ask your host what info is in the log.

If all else fails then i would sugest commenting out all of the lines
in the htaccess, testing your site then puting each rule back in and
testing again. Long winded, but at least you should find the offending
rule/s. Commenting done by putting # at the beginning of the line.

Its not a Google issue, as browsers tend to be a lot more flexable
about what they follow (so these pages work for users), but a bot is a
lot more fussy about it being correct so its not working for G.

Hello Sonnie,

this is not affected by .htaccess because there are no redirect rules in it at all (if you are using default .htaccess from vBSEO package) - every redirect is done with vBSEO php code.
Also, it is only required to check it with http header viewer to know whether redirect works correctly, for instance:
View HTTP Request and Response Header

Perhaps it would be better to contact google directly for support? I'm not sure what is the best way for this regarding this issue.