Managing multi-VB site with Sitemap and robots.txt

I'm using a vB integration that works between three sites. I have a questions regarding two of the sites.

Site A is old and established. It's ranking well, so I'm using that to secure positions for the new keyword (let's call it keyword B). Visitors to Site A or Site B have access to 85% of the forums related to [keyword B]. This allows me to gain members looking for keyword B who are finding it via Site A, while establishing credibility at Site B. Using permissions Site B also does not show [keyword A] related forums.

I've created a few [site b keyword] forum categories that are only accessible on Site B's forums. I would like to provide Google's robots access to the [site b only] forums, so that I can build up some SERPS. Up until now, I've got robots.txt blocking all access to Site B's forums (since Google is seeing 85% of them on Site A and I'm rightfully worried about duplicate content).

I have Sitemap installed on Site A.

I have a few questions on how to best execute my strategy:

1. Is there a way in vBSEO Sitemap to not include and/or exclude specific forum categories?
2. Will Sitemap generate URL's from forum categories on Site A where permissions are all set to 'No" for Site A visitors?
2. How can I adjust my robots.txt at Site B, so that Google's robots can find some of the Site B only related forums?
3. Any suggestions on how I might monitor Google to be sure I'm accidentally guilty of duplicate content?
4. If robots.txt blocks /forum from Google, what happens if a direct link is found somwhere on my site? Does Google ignore it if it's within the /forum directory? I would like to have a list of recent forum topics on my homepage on Site B.


Thanks in advance for any assistance and/or advice.

Sorry if this seems real confusing, so I'm not surprised by the lack of responses. Hoping to scale down the question and get some feedback from the experts here at vBSEO.

Two forums. One is 7 years old and it's about 'widgets'. Company who produces 'widgets' came out with 'newkeyword widgets'. Created a second site solely for 'newkeyword widgets'. Multi-site forum integration allows discussion of 'newkeyword widgets' on both sites.

Older site is ranking well for 'newkeyword widgets'. It's related and age of site helps. I would eventually like to transfer those rankings to forum that deals solely with 'newkeyword widgets'.

Here's the dilemma. I've set up robots.txt on new site to block /forums/ for the 'newkeyword widgets' forum to avoid duplicate content issues. Remember, I'm ranking well for those terms on the old site and the 'newkeyword forums' are accessible there too.

Here's what I'm considering:

• Create category that appears only on 'newkeyword widgets' forum and modify robots.txt to allow access to that forum category. This will hopefully generate some forum rankings for the new site.
• Figure out a way to move my ranking from old site to new site. Problem is I would like to keep discussion of 'newkeyword widgets' on the old site. If I do a pure 301 redirect, I lose the ability to allow multi-site discussion.

I don't want to do anything to adversely affect my current ranking of 'newkeyword widget' on old site, but need to find a way to get the new site ranking for those terms.

Thanks in advance.

Is this an accurate summary? You are trying to share the content without incurring any problems with duplicate content.



Site A - old & established; ranking well; secured ranking for keyword x; has sitemap installed

Site B - has 85% to same sub-forums as site A (for keyword x); blocked by robots.txt

Exactly Joe.

• For shared sub-forums, you will make the decision as to where you want them indexed.
• Right now, I assume the only shared forums are the 85% (for keyword x).
• So you'll block those with robots.txt on the forum where you do NOT want them indexed by the search engines.
• Additionally, you will "exclude" those forums in your sitemap installed on the forum where you do NOT want them indexed by the search engines (a vBSEO Sitemap option; see your vBulletin AdminCP).
• If the sub-forum directories are blocked using robots.txt then most search engines, including Google, should respect that even if they find a link on your site.
• You can keep an eye on your Google Webmaster Tool to check each sites for error messages, and monitor your traffic stats using Analytics (or the vBSEO Livestats plugin).
• So it is possible for you to have segments (groups of sub-forums) on both Site A and Site B indexed (i.e not blocking either site completely) if you are careful about setting up the robots.txt and sitemap settings.

Of course, these comments are based on your description. I have not see the 2/3 sites you are referring to. The more you get involved with complex setups like this, the more you set yourself up for possible issues that are overlooked accidentally or unforeseen. So please be careful.

If I move the keyword x forums exclusively to Site B and set up a 301 redirect, will I retain my top position?

That's my biggest concern. I'm not interested in ranking on both sites. I'd much rather spend my time optimizing one single site for keyword x.

You will most likely see fluctuations, and if you were flagged for any reason it could be dangerous.

However, if Google does respect the 301s you should be fine. Normally, I would say yes, you may see some fluctuations, but expect it to be restored after temporary flux.

Due to the advanced nature of your setup there are no guarantees. Long term, I think if the content is suitable for one site, I think it's best to get it all on one. The earlier the better as you don't want to see continued indexing on two different forums. In other words, you might want to take the risk now, so you'll be safe letter.

This decision should be weighed carefully depending on how much actual traffic you are already getting to the forum you want to close down.

What's the most effective way to redirect an existing forum category and subsequent threads in Site A to Site B?

BTW, thanks for all your help.

You could do it with vBSEO's Custom Redirect feature. If you are closing the full site down, it's best to do it directly within the .htaccess.