Results 1 to 14 of 14

How long does it take for "Index New Posts in Search Engine" set to "No" to take effect?

This is a discussion on How long does it take for "Index New Posts in Search Engine" set to "No" to take effect? within the General Discussion forums, part of the vBulletin SEO Discussion category; There are certain forums that I have which I would not like to appear in search engine search results. Apparently, ...

  1. #1
    Member
    Real Name
    bfseo
    Join Date
    Mar 2011
    Posts
    73
    Liked
    0 times

    How long does it take for "Index New Posts in Search Engine" set to "No" to take effect?

    There are certain forums that I have which I would not like to appear in search engine search results. Apparently, several of these have been indexed. A few days ago I discovered that I had the "Index New Posts in Search Engine" setting set to "Yes." I've since updated the setting to "No."

    Now I have a few questions.

    1. How long before search engine search results don't yield any results for the forums I've applied this setting to?

    2. Is there any way to expedite this process?

  2. #2
    vBSEO Staff Brian Cummiskey's Avatar
    Real Name
    Brian Cummiskey
    Join Date
    Jul 2009
    Location
    btwn NYC and Boston
    Posts
    12,789
    Liked
    657 times
    Blog Entries
    2
    That setting is for internal search (ie, search.php), not google/etc.

    If you don't want bots in it, you should add these forums to your robots.txt file and/or make them not viewable by guests.

  3. #3
    Member
    Real Name
    bfseo
    Join Date
    Mar 2011
    Posts
    73
    Liked
    0 times
    1. Simply making that forum unviewable to guests will help prevent it from being indexed in search engines?

    2. How would I add a specific forum to my robots.txt file if the forum url looks something like this under my current VBSEO settings?

    http://www.mydomain/forums/f29/

  4. #4
    vBSEO Staff Brian Cummiskey's Avatar
    Real Name
    Brian Cummiskey
    Join Date
    Jul 2009
    Location
    btwn NYC and Boston
    Posts
    12,789
    Liked
    657 times
    Blog Entries
    2
    1) yes, bots are guests. If you block guests, you block bots from seeing it to.

    2) Tutorial how to make a good robots.txt file - vBulletin SEO Forums
    and you would use

    Disallow: /forums/f29/

    format

  5. #5
    Member
    Real Name
    bfseo
    Join Date
    Mar 2011
    Posts
    73
    Liked
    0 times
    Quote Originally Posted by Brian Cummiskey View Post
    1) yes, bots are guests. If you block guests, you block bots from seeing it to.

    2) Tutorial how to make a good robots.txt file - vBulletin SEO Forums
    and you would use

    Disallow: /forums/f29/

    format
    Thanks!

    Same question again from earlier. How long before those forums gets removed from search engine results?

  6. #6
    vBSEO Staff Brian Cummiskey's Avatar
    Real Name
    Brian Cummiskey
    Join Date
    Jul 2009
    Location
    btwn NYC and Boston
    Posts
    12,789
    Liked
    657 times
    Blog Entries
    2
    There's no set time. When ever google decided that its a permanent dead url to them.

  7. #7
    Member
    Real Name
    bfseo
    Join Date
    Mar 2011
    Posts
    73
    Liked
    0 times
    Thanks. BTW, will this work to disable my smilies page from being indexed?

    Disallow: /forums/misc.php?do=showsmilies

  8. #8
    vBSEO Staff Brian Cummiskey's Avatar
    Real Name
    Brian Cummiskey
    Join Date
    Jul 2009
    Location
    btwn NYC and Boston
    Posts
    12,789
    Liked
    657 times
    Blog Entries
    2
    yes

  9. #9
    Member
    Real Name
    bfseo
    Join Date
    Mar 2011
    Posts
    73
    Liked
    0 times
    Thanks Bryan. You're always so helpful.

    Last question for now. Is it possible to block the bots from viewing a certain page? If so, would I block it like this...?

    Disallow: /forums/f17/forum page/

  10. #10
    vBSEO Staff Brian Cummiskey's Avatar
    Real Name
    Brian Cummiskey
    Join Date
    Jul 2009
    Location
    btwn NYC and Boston
    Posts
    12,789
    Liked
    657 times
    Blog Entries
    2
    Yup, you can also use * to anything like

    Disallow: /forums/f17/*

    which will allow /f17/ but nothing as a child of /f17/______

  11. #11
    Member
    Real Name
    bfseo
    Join Date
    Mar 2011
    Posts
    73
    Liked
    0 times
    Quote Originally Posted by Brian Cummiskey View Post
    Yup, you can also use * to anything like

    Disallow: /forums/f17/*

    which will allow /f17/ but nothing as a child of /f17/______
    So that will allow indexing of the main forum page for /f17 but not for the pages within each thread?

  12. #12
    vBSEO Staff Brian Cummiskey's Avatar
    Real Name
    Brian Cummiskey
    Join Date
    Jul 2009
    Location
    btwn NYC and Boston
    Posts
    12,789
    Liked
    657 times
    Blog Entries
    2
    correct, or anything else that's a sub-page of f17 including pagination of the forumdisplay, and the threads within f17 (assuming you use /f17/thread-123/ style urls where the forum is a part of the thread url)

  13. #13
    Member
    Real Name
    bfseo
    Join Date
    Mar 2011
    Posts
    73
    Liked
    0 times
    Is there a particular reason why someone would want the main sub-page indexed but not the thread content?

  14. #14
    vBSEO Staff Brian Cummiskey's Avatar
    Real Name
    Brian Cummiskey
    Join Date
    Jul 2009
    Location
    btwn NYC and Boston
    Posts
    12,789
    Liked
    657 times
    Blog Entries
    2
    Probably not. I was merely using that as an example

Similar Threads

  1. Replies: 3
    Last Post: 02-20-2011, 07:04 PM
  2. Tracking "clicks" on rewritten search queries like "new posts"
    By c20h25n3o in forum Custom Rewrite Rules
    Replies: 5
    Last Post: 09-13-2010, 05:29 PM
  3. does "disallowing copy" effect search engines bots ?
    By khalid890 in forum General Discussion
    Replies: 2
    Last Post: 10-14-2007, 11:29 AM
  4. Replies: 5
    Last Post: 09-09-2006, 04:11 AM
  5. Custom rewrite rule for "Search" and "Search Result"
    By Sony in forum Custom Rewrite Rules
    Replies: 1
    Last Post: 09-24-2005, 09:38 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •