Results 1 to 6 of 6

Robots.txt for forums & vBSEO

This is a discussion on Robots.txt for forums & vBSEO within the General Discussion forums, part of the vBulletin SEO Discussion category; Hi All, I logged into my adsense account and found out that the 'Robots.txt' is blocking visits. However, I am ...

  1. #1
    Senior Member
    Real Name
    Kaustubh
    Join Date
    Dec 2007
    Posts
    152
    Liked
    0 times

    Unhappy Robots.txt for forums & vBSEO

    Hi All,

    I logged into my adsense account and found out that the 'Robots.txt' is blocking visits. However, I am unable to find the robots.txt for the forums.

    Where is it?

  2. #2
    vBSEO Staff Brian Cummiskey's Avatar
    Real Name
    Brian Cummiskey
    Join Date
    Jul 2009
    Location
    btwn NYC and Boston
    Posts
    12,789
    Liked
    657 times
    Blog Entries
    2
    vb/vbseo do not ship with this file. If you made one, it was made by yourself. it ONLY functions in your site root...
    site.com/robots.txt

  3. #3
    Senior Member
    Real Name
    Kaustubh
    Join Date
    Dec 2007
    Posts
    152
    Liked
    0 times
    This is what I have: -

    User-agent: Mediapartners-Google
    Disallow:
    User-agent: *
    Disallow: /administrator/
    Disallow: /cache/
    Disallow: /components/
    Disallow: /images/
    Disallow: /includes/
    Disallow: /installation/
    Disallow: /language/
    Disallow: /libraries/
    Disallow: /media/
    Disallow: /modules/
    Disallow: /plugins/
    Disallow: /templates/
    Disallow: /tmp/
    Disallow: /xmlrpc/

  4. #4
    Senior Member webmastersitesi's Avatar
    Join Date
    Oct 2007
    Posts
    518
    Liked
    16 times
    Blog Entries
    3
    It stays at root url like sitename.com/robots.txt if you have a 404 page he tries t read content and if the bot sees any words like disallow // obey it rule.

    I think adsense bot try to enter /template/and it's blocked? What's the url it's blocked can you check?

  5. #5
    vBSEO Staff Brian Cummiskey's Avatar
    Real Name
    Brian Cummiskey
    Join Date
    Jul 2009
    Location
    btwn NYC and Boston
    Posts
    12,789
    Liked
    657 times
    Blog Entries
    2
    mediapartners bot is for adsense only. you are not blocking any crawler robots.

  6. #6
    Member WiseHYIP's Avatar
    Real Name
    Mike
    Join Date
    Oct 2010
    Location
    Belgium
    Posts
    51
    Liked
    19 times
    Seems like there is nothing blocked in there where you should lose Adsense income.

Similar Threads

  1. Robots.txt like vBSEO
    By zems in forum General Discussion
    Replies: 14
    Last Post: 07-13-2009, 01:31 PM
  2. Redirecting /forums/robots.txt to /robots.txt - Is it good?
    By MadK in forum Custom Rewrite Rules
    Replies: 6
    Last Post: 08-22-2008, 06:29 PM
  3. Replies: 4
    Last Post: 04-08-2008, 06:03 AM
  4. robots.txt and vBSEO
    By Silmarillion in forum General Discussion
    Replies: 9
    Last Post: 10-08-2007, 03:24 PM
  5. robots.txt und vbseo
    By Metro Man in forum Deutsch
    Replies: 18
    Last Post: 09-03-2007, 11:17 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •