---

[ April Full List | Reply ]

---

What is crawler.html/crawlers.html? / How 2 use them?? From WP1 on 21 April '01 Hello, Where can I get some info on "crawler.html" and/or "crawlers.html". 1. What exactly are they?? 2. Is it cool to use them?? (that is, do you get penalized by Search Engines for using them?) 3. Can it be used with "robots.txt" and "<META> tags?? Any and all information appreciated. WP

Re: What is crawler.html/crawlers.html? / How 2 use them?? - Dale - 21/04/01

Reply to this message
Required Details
Your name:
Subject:

Optional Details
Home page title:
Home page URL:
Email:	(if you'd like to receive automatic replies by email)
Your Message
>Hello, > >Where can I get some info on "crawler.html" and/or "crawlers.html". > >1. What exactly are they?? >2. Is it cool to use them?? (that is, do you get penalized by Search Engines for using them?) >3. Can it be used with "robots.txt" and "<META> tags?? > >Any and all information appreciated. > >WP