Results 1 to 4 of 4

Why do spiders do this?

This is a discussion on Why do spiders do this? within the General Discussion forums, part of the vBSEO SEO Plugin category; I know my forum is new, and probably is not worth the spiders attention, but I try to create daily ...

  1. #1
    Senior Member
    Real Name
    gotlinks
    Join Date
    Jun 2006
    Posts
    202
    Liked
    5 times

    Post Why do spiders do this?

    I know my forum is new, and probably is not worth the spiders attention, but I try to create daily content, and add new articles weekly, and from time to time I see
    mr google, yahoo, some spider99, and some spider it looks like it
    would be from overseas. They come, they go...

    now, why is it they come and go fairly quickly, but yet when I
    run across some old inactive forum, well, the only member
    activity they seem to have is spiders, and no matter what
    time of day you go there, you will see spiders on the forum...

    I would love to have an active forum with little spiders
    rather then a dead forum with 25 spiders....


    Whats going on with this?

  2. #2
    vBSEO Staff Brian Cummiskey's Avatar
    Real Name
    Brian Cummiskey
    Join Date
    Jul 2009
    Location
    btwn NYC and Boston
    Posts
    12,789
    Liked
    657 times
    Blog Entries
    2
    As you have more content and more frequent changes, they will stay on longer. As you get better indexed, the bots will go out to check for updates. Things just don't happen over night, and bots are 100% out of anyones control bot the bot owner.

  3. #3
    Senior Member
    Real Name
    gotlinks
    Join Date
    Jun 2006
    Posts
    202
    Liked
    5 times
    Quote Originally Posted by Brian Cummiskey View Post
    As you have more content and more frequent changes, they will stay on longer. As you get better indexed, the bots will go out to check for updates. Things just don't happen over night, and bots are 100% out of anyones control bot the bot owner.
    that I understand..but why are bots all over dead forums?
    its not like they havent seen the content million times
    already...dead forums maybe get a post here and there, but
    not the need for a bot to stay on the forum day and night,
    this is what I do not get.

  4. #4
    vBSEO Staff Brian Cummiskey's Avatar
    Real Name
    Brian Cummiskey
    Join Date
    Jul 2009
    Location
    btwn NYC and Boston
    Posts
    12,789
    Liked
    657 times
    Blog Entries
    2
    even a dead forum appears different to bots.

    today/yesterday turns into a date
    brithdays change
    etc etc

    the page has different code on it

Similar Threads

  1. where are the spiders?
    By obx11 in forum General Discussion
    Replies: 8
    Last Post: 03-03-2007, 11:17 PM
  2. Spiders!
    By sohan in forum Analysis: Traffic & SERPS
    Replies: 5
    Last Post: 11-11-2006, 06:40 PM
  3. How Many Spiders...?
    By Atheda in forum General Discussion
    Replies: 5
    Last Post: 05-18-2006, 04:44 PM
  4. Question about spiders.
    By K4n4Dian-H4x0r in forum Off-Topic & Chit Chat
    Replies: 8
    Last Post: 11-17-2005, 11:08 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •