Results 1 to 4 of 4

Sitemap generator for Google on large sites

This is a discussion on Sitemap generator for Google on large sites within the Pre-Sales Questions forums, part of the Announcements & Pre-Sales category; My site has more than 400,000 posts. Does the Google Sitemap Generator work correctly for sites of that size? Are ...

  1. #1
    Ace
    Ace is offline
    Junior Member Ace's Avatar
    Real Name
    Ace
    Join Date
    Nov 2007
    Location
    Nelson, NZ
    Posts
    26
    Liked
    0 times

    Sitemap generator for Google on large sites

    My site has more than 400,000 posts. Does the Google Sitemap Generator work correctly for sites of that size? Are there any potential problems with page timeouts, etc?

    Which pages are included in the sitemap? Is it just posts, or are the forum / summary pages included too?

    How is the sitemap frequency and priority information set?

    Does having a sitemap really help get any more pages into the Google index? It seems like their crawler is pretty good at finding pages on its own. Do you have any concrete user feedback in this area?

  2. #2
    Senior Member
    Real Name
    Keith Cohen
    Join Date
    Jul 2005
    Location
    Raleigh, NC USA
    Posts
    6,147
    Liked
    12 times
    We have sites much larger than that running it fine. It has a setting where you can add a delay between each sitemap file, to keep your server load from going too high.

    You can choose what is included, Form URLs, Thread URLs, Post URLs, etc, along with a lot of other settings.

    Yes, having a sitemap is imperative.

  3. #3
    Ace
    Ace is offline
    Junior Member Ace's Avatar
    Real Name
    Ace
    Join Date
    Nov 2007
    Location
    Nelson, NZ
    Posts
    26
    Liked
    0 times
    So an individual thread with 50,000 posts wouldn't be a problem?

    What are the options for how frequency and priority are set?

    I've heard several people say that a sitemap is "imperative", but I'm not sure if that's a result of Google propaganda or real-life experience. I've never heard any concrete feedback on the before/after results of adding one to a large site. Do you have any stories from your customers that you might be able to pass along?

  4. #4
    Senior Member
    Real Name
    Keith Cohen
    Join Date
    Jul 2005
    Location
    Raleigh, NC USA
    Posts
    6,147
    Liked
    12 times
    First, I would never allow any thread to have 50,000 posts on it. Even Jelsoft suggests against letting them get that large. It will be bad for system performance. But, it shouldn't be a problem for the sitemap generator as long as your server can handle it.

    I don't work for Google, so it can't be propoganda. I have first hand experience with many sites where a sitemap has made a huge difference in how fast, and how well, a site was reindexed. Why would you want Google to just randomly wander around your site trying to find things when you can give it a roadmap so that it can directly start indexing the "good stuff"?

Similar Threads

  1. vBSEO Sitemap Generator 2.0 Released
    By Juan Muriente in forum vBSEO Announcements
    Replies: 18
    Last Post: 12-11-2007, 02:44 PM
  2. Replies: 0
    Last Post: 12-17-2005, 02:11 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •