deadlock Message Board
[ June Full List | Reply ]

Re: HELP! Need info on SE spiders....
From David A on 26 June '00
replying to HELP! Need info on SE spiders.... posted by Brett LaCroix

>I just discovered a huge problem... I have a brand new server... and I configured it using a virtal name aliasing system that works very well.
>
>If I want to create a subdomain, I simply create a folder and request the page... no configuration needed.
>
>The only downfall is that it only works with browsers that use the HTTP/1.1 protocol, because it needs to know the full address that the browser is looking for.
>
>I did not think this would be a problem, since the last few versions of IE and Netscape support this protocol.
>
>I didn't even think about SE spiders!!!!
>
>I already know that Lycos is still using HTTP:/1.0, and thus cannot find any of my pages on my server.
>
>Does anyone know where I can find info on SE spiders and find out which one's, if any, are using the new HTTP:/1.1 protocol???
>
>This is a HUGE problem... and my access and error logs are showing hit after hit resulting in pages not found... all from SE spiders.
>
>Any info or direction to other info sites would be appreciated.
>
>Thanks,
>
>Brett

Everything and more

http://www.spiderhunter.com/




Reply to this message

Required Details
Your name:
Subject:

Optional Details
Home page title:
Home page URL:
Email:
(if you'd like to receive automatic replies by email)

Your Message