vBulletin SEO Forums

SEO

vBulletin Search Engine Optimization

Buy vBSEO Now! HACKER SAFE certified sites prevent over 99.9% of hacker crime.
ne nw
vBSEO Total Support Team Launches DeskPro New vBSEO Discount Level for Network Builders vBSEO 3.2.0 GOLD Has Landed Success with vBSEO = 600ore Web Visitors + $1400 in a Day! Crawlability Inc. Files for SEO Technology Patent
se sw

About robots.txt

This is a discussion on About robots.txt within the General Discussion forums, part of the vBulletin SEO Discussion category; Hello everyone, I want to block certain threads manually via robots.txt and I have a question doing so: eg. if ...

Go Back   vBulletin SEO Forums > vBulletin SEO Discussion > General Discussion

Enhancing 80 million pages.

Register FAQ Members List Social Groups Calendar Search Today's Posts Mark Forums Read
  #1  
Old 02-18-2008, 12:06 PM
Junior Member
 
Join Date: Jan 2006
Posts: 28
About robots.txt

Hello everyone,
I want to block certain threads manually via robots.txt and I have a question doing so:
eg. if I want to include a thread with several pages into robots.txt like this:
www.mysite.com/forums/1234-testthread.html
www.mysite.com/forums/1234-testthread-2.html
www.mysite.com/forums/1234-testthread-3.html

is it possible to have a syntax like this to include all pages of the thread without typing everyone?
www.mysite.com/forums/1234-*

I tried this one but without success.
Can someone help me out?

Thank you so much in advance
Diver
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #2  
Old 02-18-2008, 08:18 PM
briansol's Avatar
Senior Member
vBSEO Pre-Release TeamDesign for SEOBig Board Administrator
 
Real Name: Brian
Join Date: Apr 2006
Location: Central CT, USA
Posts: 5,545
the * should work. but you shouldn't include your .com. just relative path from root.

not sure if you can end in a *, but the * is valid.

try
Disallow: /forums/1234-testthread*.html


-=-------------

also, keep in mind that it may take a month before this gets acknowledged. it's certainly not an instant process.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads

Thread Thread Starter Forum Replies Last Post
robots.txt Zenith General Discussion 46 10-21-2008 02:00 PM
robots.txt wolf32 General Discussion 0 06-30-2007 10:29 PM
Robots.txt help Dave Hybrid General Discussion 2 05-29-2007 11:16 AM
No Robots.txt friendly Troubleshooting 1 10-26-2006 11:30 AM
robots.txt croportal General Discussion 1 02-05-2006 01:31 PM


All times are GMT -4. The time now is 01:29 AM.


Powered by vBulletin Version 3.8.0 Beta 4
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.5 ©2008, Crawlability, Inc.