deadlock Message Board
[ June Full List | Reply ]

Re: cgi-bin
From Bob on 12 June '00
replying to cgi-bin posted by peter laurie

>Hi
>Our site runs entirely from scripts that live in a cgi-bin directory.
>I understand that some spiders don't like to see 'cgi-bin' in a URL, so I've aliased this to 'bin'. Does anyone know if that would put them off?

Are you using robots.txt and crawler.html to tell the spiders where and where not to index? That may help. Search on those terms.




Reply to this message

Required Details
Your name:
Subject:

Optional Details
Home page title:
Home page URL:
Email:
(if you'd like to receive automatic replies by email)

Your Message