Google Webmaster Tools - Recommended for All vBSEO Customers

Google Webmaster Tools is provided by Google to allow webmasters to manage their sitemaps and evaluate their site from an SEO perspective, with direct feedback from Google:
https://www.google.com/webmasters/tools/

Many of our customers are using this already. Those of you who are not should signup for the Google Webmaster Tools ASAP.

Here is some of the interesting things that you can and should be doing with your account.

Diagnostics / Web crawl

Check this section to find errors that occurred while Google was crawling your site. When you find them, fix them. This can help you identify problematic pages on your site.

Diagnostics / Content Analysis

This area can help you flag problems with title tags or meta tags, including duplicates, those that are too long, too short, or not sufficiently descriptive.

Statistics / Top Search Queries

The most interesting part of this section is that it will show you the top queries done at Google for which your site is returned in the results.

It shows you how often your site is a result for each term, regardless of whether or not the user later clicks.

Tip:

• Is there a term that is getting ranked, but is not getting clicked? Try optimizing the title of the page. Keep the term that it ranks for, but make the title more compelling to users to help get them to click through more readily. Search engine friendly, content rich URLs, as provided by vBSEO can help improve click through.

Statistics / What Googlebot Sees

Find out what anchor text is being picked up from the anchor text in links to *your* site, from *other* sites.

Tips:

• If you are trying to rank for a particular term, and it's not included in this list, then you need to start building links to your site that specify contain your target keyword (where possible).

Statistics / Crawl Stats

Here Google summarizes your PageRank distribution. Don't be alarmed when you find that the large majority of your pages are listed as low or not yet assigned. This is common with forums.

If 1% of your forum pages have a high pagerank (if this equates to link value), then you can do very well. Don't expect all 10 million of your posts to achieve top link value.

Links / Pages with External Links

This will show you all of your pages that have external links to them, including how many, and the exact links.

Tips:

• Use this to find out if Google has discovered the new links you have created for particular pages.
• If you have links that are yet undiscovered you might expect some future ranking improvement.
• If you purchased links and they are not getting listed, then save your money next time. The site providing the links may be penalized or de-listed by Google and not worth your investment.

Links / Pages with Internal Links

This will show you all of your pages for which Google has found internal links (that is, links from your own site) to them.

Tips:

• Download the data. Sort it order of most internal links to least. If you find that your main pages are not showing up there may be a problem with your link structure. Make sure your top priority pages are showing up with internal links. If you don't have internal links to a page you are trying to optimize, create some asap!

Links / Sitelinks

If you are lucky enough to discover that you have site links, then you know that you have Google's attention. Make sure you make the most of it, and optimize your site as pro actively as you can.

Sitemap

Make sure you include your sitemap. If you did not yet install the vBSEO Sitemap Generator (different from vBSEO itself), you should do so asap.

Once you have it in place, check this section of the control panel to ensure that there are no errors, and also to track when it was downloaded and how many URLs were discovered.

Tools / Analyze robots.txt

If you have a robots.txt file in place, make sure that it is working. Check the status in this section. Find out if there are any errors.

Tools / Manage Crawl Rate

Although we want Google to crawl and index our site as quick as possible, some sites have limited server resources.

In cases like this, you might want to select a slower crawling rate to prevent your server or hosting account from getting overwhelming during peak crawling times.

In general, if your server can handle it, let Google crawl your forum as fast as it can possibly do so, without any limitations.

Tips:

• The handy graphs that are provided are a great way for you to see how much attention Google has been paying to your site, at a moment's glance. Expect fluctuations, but if you see a rapid decline, followed by a period of low to no crawling activity, something may be wrong. You have to check your site for errors.
• Check the # of kbs per day downloaded to get an idea of how much bandwidth is being consumed. For most of you this is not a big concern. Bandwidth is cheap today. Server resources are what come at a greater premium.
• Check the time spent download a page, if this is consistently averaging in excess of 1,000+ milliseconds, it could indicate that your server resources are getting over taxed.

Tools / Set Geographic Target

If your site is intended for a Global audience, you don't need to change this setting.

If you are focused on a particular country or region, then select this setting.

Tip:

• Don't be fooled by the promise of "global traffic". If your site is in a language spoken primarily in one small region of the world, then optimize for that. You could do better by focusing then trying to target an entire world of people who will never visit your forum.

Tools / Set preferred domain

If you are using www. or non-www versions of your site, or any other version, you should let Google know which one to use here.

If you ever change the domain used, always remember to update Google.

And keep in mind that you should have a 301 redirect in place on your site that ensures that any traffic coming in to the old location is instantly forwarded to the new location.

Tools / Set enhanced image search

Google has an interest new project on using humans to actually classify your images manually.

If you are a forum that has a lot of photos or a gallery, I think it would be worthwhile turning this feature on.

You also might find it fun to procrastinate playing the Google Image Labeler game.

Tools / Remove URLs

If you have any URLs that you want removed for any reason, this is the place to do it.

Note: Google also expects you to physically remove them, block them with robots.txt or apply noindex code within the pages as well.

Thanks for the reminder! That's some good article you got there, Joe.

In my dashboard i have domain.com and domain.com/forum is this correct or should i delete domain.com/forum?

Unless you have two seperate sitemaps, there probably isn't a need to have both.

For example, i have article software that creates a sitemap, and i also have the forums which create a different sitemap. Therefore i've got a similar situation as yours Julio.

Originally Posted by Julio

In my dashboard i have domain.com and domain.com/forum is this correct or should i delete domain.com/forum?

Do you mean it lists multiple sites, or multiple sitemaps?

If sites, you only need the main forum URL.

Thank you both!

EDIT: I'm comsufed now! because i have 2 different sitemaps. One for domain.com and one for domain.com/forum

Do i delete sitemap from domain.com will sitemap from domain.com/forum scan the pages of domain.com?

Our vBSEO sitemap will only handle forum URLs, unless you *manually* add others to the extra-urls.txt file.

Thanks Joe,

In what format do i need to enter the urls in the file?

You can either simply enter URLs one per line or add custom priority/frequency for them:

Code:

http://www.domain.com/page1.html,daily,0.9
http://www.domain.com/page3.html,weekly,0.5

Statistics / Top Search Queries

The most interesting part of this section is that it will show you the top queries done at Google for which your site is returned in the results.

It shows you how often your site is a result for each term, regardless of whether or not the user later clicks.

Tip:

• Is there a term that is getting ranked, but is not getting clicked? Try optimizing the title of the page. Keep the term that it ranks for, but make the title more compelling to users to help get them to click through more readily. Search engine friendly, content rich URLs, as provided by vBSEO can help improve click through.

First of all thanks for this very useful thread.

After reading this I find out a lot of things went wrong with my site and now fixing those.

However re: above quoted something strange happening, in Webmaster tools it is showing mysite on SERP 4 and when I click on link it shows my site on 7 location on first page for term "urdu poetry", But when I open a new browser and search for same term, my site (Urdu Poetry Community & Forum) appear on 2nd page on position 11-12.

Any idea whats wrong here ?

Thanx Joe Ward
i added xml site map.....is it better than the html sitemap such as vbseo forum sitemap....(forum.com/sitemap/) ??

also google found about 21 url error in my xml site map
it says( URL not allowed
This url is not allowed for a Sitemap at this location)

and what about our sitemaps like this: forum.com/sitemap_index.xml.gz
should i add only it?
or i have to add all sitemap pages such as sitemap_1.xml.gz,sitemap_2.xml.gz,sitemap_3.xml.gz ?

I use and great thing surely

Originally Posted by Basit

First of all thanks for this very useful thread.

After reading this I find out a lot of things went wrong with my site and now fixing those.

However re: above quoted something strange happening, in Webmaster tools it is showing mysite on SERP 4 and when I click on link it shows my site on 7 location on first page for term "urdu poetry", But when I open a new browser and search for same term, my site (Urdu Poetry Community & Forum) appear on 2nd page on position 11-12.

Any idea whats wrong here ?

There may be some time lag in stats generation, discrepancies between data centers, and some averaging that will not reflect real time situations.

thank you

Hi Joe,

Great thread.

Google says this about my site:

You have not submitted any Sitemaps. Submit a Sitemap to help Google discover pages our crawlers might not otherwise find.
Once you create and submit a Sitemap listing the URLs on your site, we'll provide you with data on how Google is indexing those pages. More information

....I have sitemap installed and it pings my site info to the search engines every day though? :S

...Can you help?