---

[ June Full List | Reply ]

---

Re: meta commands for robots From Hope on 14 June '01 replying to meta commands for robots posted by jwj The command Index, Follow give an open invitation to the spiders to go through your entire site. Not all spiders follow this tag. You should use robots.txt for better control over a spiders move over your site. >As DRB suggested, > >"<META NAME="ROBOTS" CONTENT="INDEX, FOLLOW"> >The last line for Robots is an Important line to use." > >I do realize that the commands NOT to follow are important when they apply, but do you guys know whether there is any level of significance to the command to index and follow? Or might that be of no effect?

Reply to this message
Required Details
Your name:
Subject:

Optional Details
Home page title:
Home page URL:
Email:	(if you'd like to receive automatic replies by email)
Your Message
>The command Index, Follow give an open invitation to the spiders to go through your entire site. Not all spiders follow this tag. You should use robots.txt for better control over a spiders move over your site. > >>As DRB suggested, >> >>"<META NAME="ROBOTS" CONTENT="INDEX, FOLLOW"> >>The last line for Robots is an Important line to use." >> >>I do realize that the commands NOT to follow are important when they apply, but do you guys know whether there is any level of significance to the command to index and follow? Or might that be of no effect?