Results 1 to 5 of 5

Bot Crawling Question

This is a discussion on Bot Crawling Question within the General Discussion forums, part of the vBulletin SEO Discussion category; Do bots such as Google, Yahoo etc crawl private forums? For example, i have certain sections of my forum that ...

  1. #1
    Member
    Real Name
    Vishal Patel
    Join Date
    May 2007
    Posts
    88
    Liked
    0 times

    Bot Crawling Question

    Do bots such as Google, Yahoo etc crawl private forums?

    For example, i have certain sections of my forum that I want people to be registered for, however I do want the bots to pick them up....can this be done??

    Thanks

  2. #2
    Senior Member
    Real Name
    dave
    Join Date
    Jun 2006
    Posts
    348
    Liked
    0 times
    Blog Entries
    1
    Quote Originally Posted by nymyth View Post
    Do bots such as Google, Yahoo etc crawl private forums?

    For example, i have certain sections of my forum that I want people to be registered for, however I do want the bots to pick them up....can this be done??

    Thanks
    Log out of your forums.

    Now whatever you can see, thats what search engine bots can see.

    You'll need a modification or something to show the bots the data thats just for registered users, however that defeat the purpose as if the bots can see it, they'll cache it. Which means anyone will be able to see it.

  3. #3
    Member
    Real Name
    Vishal Patel
    Join Date
    May 2007
    Posts
    88
    Liked
    0 times
    hmmmm....thanks for that....

    another question: does vbSEO allow bots to crawl PM's ... cuz i saw a bot on WOL and it had "Viewing Conversation" as its location

  4. #4
    Senior Member briansol's Avatar
    Real Name
    Brian
    Join Date
    Apr 2006
    Location
    Central CT, USA
    Posts
    6,981
    Liked
    8 times
    conversation is NOT pm. conversation is visitor messages.

  5. #5
    Member
    Real Name
    Vishal Patel
    Join Date
    May 2007
    Posts
    88
    Liked
    0 times
    ah gotcha...thanks

Similar Threads

  1. Sitemap crawling issue
    By samel in forum Troubleshooting
    Replies: 3
    Last Post: 07-31-2007, 05:10 AM
  2. Yahoo has stopped crawling
    By JD76 in forum General Discussion
    Replies: 7
    Last Post: 07-30-2007, 03:06 PM
  3. Google Crawling - Sitemap.
    By Lasher in forum General Discussion
    Replies: 5
    Last Post: 07-29-2007, 09:15 AM
  4. Enable crawling of showpost?
    By Haakon in forum General Discussion
    Replies: 6
    Last Post: 01-06-2006, 12:54 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •