Twing

The configurability of our URLs makes it difficult for a standard crawler checking for a 1-size-fits-all format.

I don't see what's hard about it... a CRAWLER does just that... it crawls links.

it's all of 10 lines of code in javascript...

Code:

var links = document.getElementsByTagName("a");
      for (var i = 0; i < links.length; i++ ) {
            // ajax call to some bot crap here
      }

wheter it's /a/nice/url or showthread.php?t=213, accessing the link is the same process.

With dynamic links they can tell exactly what each type of page it is (showthread, etc), and they may only be interested in a certain type. A search engine does not distinguish between page type. So for those interested in specific page types, it requires a little more work to confirm that a page is actually the type of page they want (with varying format, re-written, static URLs).

Originally Posted by tavenger5

I spoke to Jon from Twing at web 2.0 expo. He explained that they can't currently crawl vbulletin forums with vbseo installed, but sent me a script that will generate a map their software can use to import threads (it pretty much pulls thread id's right from the database, then they index the rewritten URL)

Question is, think Twing is worth it? They remind me of boardreader, but cleaner.

Hey, it was good meeting you at Web 2.0!

My information was erroneous, as we currently are fully vBSEO compatible, and our crawler handles vBSEO forums. We have vBSEO Forums in our index. If you want to get your Forum included (in case it is not already), you can just go to our Forum Submission Page: Twing - Submit Forum

Originally Posted by Keith Cohen

That sounds like another feature to highlight about vBSEO! Now if we could prevent Boardreader and the like, we'd be set.

I hate those types of sights... I don't want others profiting from my site's content.

Hmmm, isn't the point of vBSEO to get your Forum optimized for Search Engines?

We're a Forum Search Engine. We drive traffic to your site. We don't rip off your content (just show a snippet like a normal search engine result), nor do we make money from your site at all (just some query relevant ads on the side our search results page).

Of course, if you don't want to be in Twing, that's totally cool, too. We respect robots.txt, and you can just block our crawler: (Twingbot/1.0 (+Twing Crawler)). I'd be happy to handle any other problems or questions you may have.

Originally Posted by briansol

I don't see what's hard about it... a CRAWLER does just that... it crawls links.

it's all of 10 lines of code in javascript...

Code:

var links = document.getElementsByTagName("a");
      for (var i = 0; i < links.length; i++ ) {
            // ajax call to some bot crap here
      }

wheter it's /a/nice/url or showthread.php?t=213, accessing the link is the same process.

Twing does some more complicated, and we think cooler, things. We understand the structure of Forums (Post/Thread/Forum), give a lot of information about the post, have a full, robust Forum Directory, all so that users can come to us and find the best Forum for them.

Our goal is to be very friendly and useful to Forum owners. Currently, it is pretty hard to surface forums using conventional search engines (which is why vBSEO was created), so we decided to create one dedicated to the wonderful world of Forums.

I just tried adding my site about 2-3 times and always got this

"Forum was not added." in a nice yellow box.

Any advice or tips on getting added

Thank-you for visiting us and providing some details John.