Results 1 to 10 of 10

Google spiders my staff area

This is a discussion on Google spiders my staff area within the General Discussion forums, part of the vBSEO Google/Yahoo Sitemap category; I see google is trying to spider my staff area I see also they getting the no permission message. but ...

  1. #1
    Senior Member Bonus Paradise's Avatar
    Real Name
    Marina
    Join Date
    Jun 2008
    Location
    Europe
    Posts
    180
    Liked
    0 times

    Google spiders my staff area

    I see google is trying to spider my staff area
    I see also they getting the no permission message.
    but still I dont like them to be there, lol

    In Vbulletin Options - vBSEO Google/Yahoo Sitemap Generator

    I see this ....
    Excluded Forums List

    There I have added 5
    which is the ID of my whole staff category
    but how i remember i did later than when I generated my sitemap
    Is this the reason why google still find this links?

    Will this be solved soon,
    does the sitemap then update and this links for Staff are then not anymore there?

    Another question,
    If google try to spider now, ok the get no permission
    will the still index maybe the title of posts?
    hope not.

    Please let me know what I have to do

    Thank you very much

    Marina

  2. #2
    vBSEO Staff Oleg Ignatiuk's Avatar
    Real Name
    Oleg Ignatiuk
    Join Date
    Jun 2005
    Location
    Belarus
    Posts
    25,689
    Liked
    157 times
    Are those URLs included in your sitemap?
    Oleg Ignatiuk / Crawlability Inc.
    vBSEO 3.6.0 GOLD Released!
    Unveiling the NEW vBSEO Sitemap Generator 3.0. - available NOW for vBSEO Customers!


  3. #3
    Senior Member Bonus Paradise's Avatar
    Real Name
    Marina
    Join Date
    Jun 2008
    Location
    Europe
    Posts
    180
    Liked
    0 times
    Quote Originally Posted by Oleg Ignatiuk View Post
    Are those URLs included in your sitemap?
    how am i able to see that Oleg?

    well as said already
    I did generate the sitemap
    and only later i have seen this setting to exclude forums
    so I added this exclude forum ID 5
    i think 2 or 3 days after I did generate the map.

  4. #4
    Senior Member Bonus Paradise's Avatar
    Real Name
    Marina
    Join Date
    Jun 2008
    Location
    Europe
    Posts
    180
    Liked
    0 times
    I try to look here
    http://www.bonusparadise.com/forum/sitemap_1.xml.gz

    I am there now
    just needed way long to load
    I will see what I find there

  5. #5
    Senior Member Bonus Paradise's Avatar
    Real Name
    Marina
    Join Date
    Jun 2008
    Location
    Europe
    Posts
    180
    Liked
    0 times
    Yes I found some posts there from my staff area
    what should i do Oleg?

    ty

    exmaple:
    Code:
     
    <url>
    <loc>http://www.bonusparadise.com/forum/bingo-banner/92-jackpot-joy-no-usa.html</loc> 
     
    <priority>0.58</priority> 
     
    <lastmod>2008-06-18T09:44:37+00:00</lastmod> 
     
    <changefreq>monthly</changefreq> 
     
    </url>

  6. #6
    vBSEO Staff Oleg Ignatiuk's Avatar
    Real Name
    Oleg Ignatiuk
    Join Date
    Jun 2005
    Location
    Belarus
    Posts
    25,689
    Liked
    157 times
    Please try to modify vbseo_sitemap_functions.php file:
    FIND:
    PHP Code:
    (isset($vboptions['hideprivateforums'])&&!$vboptions['hideprivateforums']) 
    REPLACE WITH:
    PHP Code:
    ($vboptions['hideprivateforums']||!isset($vboptions['hideprivateforums'])) 
    Oleg Ignatiuk / Crawlability Inc.
    vBSEO 3.6.0 GOLD Released!
    Unveiling the NEW vBSEO Sitemap Generator 3.0. - available NOW for vBSEO Customers!


  7. #7
    Senior Member Bonus Paradise's Avatar
    Real Name
    Marina
    Join Date
    Jun 2008
    Location
    Europe
    Posts
    180
    Liked
    0 times
    I have done that

    this is how it looks now in file

    PHP Code:
     AND ($vboptions['hideprivateforums']||!isset($vboptions['hideprivateforums']))  )
       {
        continue; 
    when and how can i see if this works?
    should the urls from sitemap disappear immediatly or how long should i wait to check?

    thanks a lot for the effort
    you guys are the best!

  8. #8
    vBSEO Staff Oleg Ignatiuk's Avatar
    Real Name
    Oleg Ignatiuk
    Join Date
    Jun 2005
    Location
    Belarus
    Posts
    25,689
    Liked
    157 times
    You should regenerate sitemap manually via vbseo_sitemap/ interface and check that those URLs are not included in sitemap now.
    Oleg Ignatiuk / Crawlability Inc.
    vBSEO 3.6.0 GOLD Released!
    Unveiling the NEW vBSEO Sitemap Generator 3.0. - available NOW for vBSEO Customers!


  9. #9
    Senior Member Bonus Paradise's Avatar
    Real Name
    Marina
    Join Date
    Jun 2008
    Location
    Europe
    Posts
    180
    Liked
    0 times
    thanks Oleg
    I did generate the sitemap again
    and i see now after the re generate in that part
    Code:
    [forumdisplay] forum_id: 1, total threads: 0, pages: 1 [1,411.8Kb mem used] [0s (+0s)]
    [forumdisplay] forum_id: 2, total threads: 99, pages: 4 [1,424.1Kb mem used] [0s (+0s)].............. 
    ...............
    there is no forum ID which belongs to my staff
    so that should be fine then hm?

    just checked the sitemap quick
    I see no forum posts or urls so far

    thanks for the help again

    Marina

  10. #10
    vBSEO Staff Oleg Ignatiuk's Avatar
    Real Name
    Oleg Ignatiuk
    Join Date
    Jun 2005
    Location
    Belarus
    Posts
    25,689
    Liked
    157 times
    Yes, it should be fine now then.
    Oleg Ignatiuk / Crawlability Inc.
    vBSEO 3.6.0 GOLD Released!
    Unveiling the NEW vBSEO Sitemap Generator 3.0. - available NOW for vBSEO Customers!


Similar Threads

  1. Just a quick one ^^ (Google Spiders)
    By Yours Truly in forum General Discussion
    Replies: 2
    Last Post: 02-29-2008, 04:27 PM
  2. Download area ?
    By sensimilla in forum Troubleshooting
    Replies: 2
    Last Post: 10-22-2007, 09:34 AM
  3. Discussion area regarding version 3
    By Mike in forum General Discussion
    Replies: 3
    Last Post: 06-01-2006, 10:55 AM
  4. weeks after google spiders, still old pages in SERPs
    By Scott Mueller in forum General Discussion
    Replies: 4
    Last Post: 04-24-2006, 07:02 PM
  5. This area is password protected.
    By Greg Watson in forum Troubleshooting
    Replies: 10
    Last Post: 10-30-2005, 04:54 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •