Results 1 to 11 of 11

Spiders stopped being logged

This is a discussion on Spiders stopped being logged within the Troubleshooting forums, part of the vBSEO Google/Yahoo Sitemap category; Dead on midnight yesterday my logs stopped filling and are now empty, no spiders are showing as having crawled any ...

  1. #1
    Senior Member Dave Hybrid's Avatar
    Real Name
    Dave Hybrid
    Join Date
    Mar 2007
    Location
    UK
    Posts
    400
    Liked
    8 times

    Spiders stopped being logged

    Dead on midnight yesterday my logs stopped filling and are now empty, no spiders are showing as having crawled any pages.

    All my other log sources suggest crawl access is normal.

    Would appreciate any advice.
    Last edited by Dave Hybrid; 07-10-2009 at 04:10 AM.

  2. #2
    Senior Member Dave Hybrid's Avatar
    Real Name
    Dave Hybrid
    Join Date
    Mar 2007
    Location
    UK
    Posts
    400
    Liked
    8 times
    I've added a pic, I don't see how it can be anything i've done as it stops dead on midnight...

    Attached Thumbnails Attached Thumbnails example.gif  

  3. #3
    vBSEO.com Webmaster Mert Gökçeimam's Avatar
    Real Name
    Lizard King
    Join Date
    Oct 2005
    Location
    Istanbul, Turkey, Turkey
    Posts
    22,367
    Liked
    542 times
    Blog Entries
    4
    Hello ,

    We are still on July 10 therefore the logs will updated when July 10 is completed
    Mert Gökçeimam / Crawlability Inc.

    vBSEO 3.6.0 Alpha Önizlemesi - Including Like Tree
    Unveiling the NEW vBSEO Sitemap Generator 3.0 - available NOW for vBSEO Customers!


    Twitter:@Depkac
    Personal Blog : Mert Gökçeimam

  4. #4
    Senior Member Dave Hybrid's Avatar
    Real Name
    Dave Hybrid
    Join Date
    Mar 2007
    Location
    UK
    Posts
    400
    Liked
    8 times
    They used to update in real time as the bots crawled, not daily. I use this all day long and am 1000% it is not normally like this.

  5. #5
    Senior Member Dave Hybrid's Avatar
    Real Name
    Dave Hybrid
    Join Date
    Mar 2007
    Location
    UK
    Posts
    400
    Liked
    8 times
    I deleted yesterdays text file log via ftp and it's all working now, must have been corrupt or something.

    Thanks.

  6. #6
    Senior Member Dave Hybrid's Avatar
    Real Name
    Dave Hybrid
    Join Date
    Mar 2007
    Location
    UK
    Posts
    400
    Liked
    8 times
    This is back and seems to be happening more frequent.

    The logs fill up in real time, then just stop dead and all data disappears, deleting the days text file restarts it.

    It's almost as the text file is being corrupted, so cannot be read/written to. Happening all day, i delete the file and it logs for a few hours, then stops.

    Please advise on how to fix this please.

    Thanks.

  7. #7
    vBSEO Staff Brian Cummiskey's Avatar
    Real Name
    Brian Cummiskey
    Join Date
    Jul 2009
    Location
    btwn NYC and Boston
    Posts
    12,782
    Liked
    648 times
    Blog Entries
    2
    Are you running out of disk space? what about /tmp ? Those are the only logical explinations to me as to why this would stop working randomly throughout the day.
    Brian Cummiskey / Crawlability Inc.
    vBSEO 3.6.0 GOLD Released!
    Unveiling the NEW vBSEO Sitemap Generator 3.0. - available NOW for vBSEO Customers!


  8. #8
    Senior Member Dave Hybrid's Avatar
    Real Name
    Dave Hybrid
    Join Date
    Mar 2007
    Location
    UK
    Posts
    400
    Liked
    8 times
    Thanks Brain. The file is being corrupted afaik, so cannot be accessed, read or written to, the bot logs just goes blank, blank data in vbseo. If I download the days file it is empty, although it was filling up for hours before. If i delete the file, it restarts until next time. Disk space is fine.

  9. #9
    vBSEO Staff Brian Cummiskey's Avatar
    Real Name
    Brian Cummiskey
    Join Date
    Jul 2009
    Location
    btwn NYC and Boston
    Posts
    12,782
    Liked
    648 times
    Blog Entries
    2
    Next time it stops writing, attach the log file here for review (even if its blank). there may be a rogue character in it somewhere.

    Also, what is your wait_timeout setting?
    Brian Cummiskey / Crawlability Inc.
    vBSEO 3.6.0 GOLD Released!
    Unveiling the NEW vBSEO Sitemap Generator 3.0. - available NOW for vBSEO Customers!


  10. #10
    Senior Member Dave Hybrid's Avatar
    Real Name
    Dave Hybrid
    Join Date
    Mar 2007
    Location
    UK
    Posts
    400
    Liked
    8 times
    Here is the file and screen, as you can see yesterday stopped again after i went to bed...

    Edit; Can't seem to attach the log file, regardless it is empty, 0 bytes, and contains no rogue characters.
    Attached Thumbnails Attached Thumbnails bots..png  

  11. #11
    vBSEO Staff Oleg Ignatiuk's Avatar
    Real Name
    Oleg Ignatiuk
    Join Date
    Jun 2005
    Location
    Belarus
    Posts
    25,689
    Liked
    157 times
    Hello,

    please try to modify functions_vbseo_seo.php file:
    find:
    if (file_exists($hfilename))
    and replace with:
    if (file_exists($hfilename) && (filesize($hfilename) > 0))
    Oleg Ignatiuk / Crawlability Inc.
    vBSEO 3.6.0 GOLD Released!
    Unveiling the NEW vBSEO Sitemap Generator 3.0. - available NOW for vBSEO Customers!


Similar Threads

  1. Users getting logged out
    By DreyerMedia in forum Troubleshooting
    Replies: 42
    Last Post: 11-18-2009, 04:34 PM
  2. getting logged out
    By Chad in forum Troubleshooting
    Replies: 1
    Last Post: 03-02-2007, 10:54 AM
  3. Users getting logged out
    By ChurchMedia in forum Bug Reporting
    Replies: 2
    Last Post: 09-14-2006, 06:58 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •