Lycos Robot and dynamic pages|
From Baz on 19 April '99
Just been checking my logs, and I noticed something strange.
I have an ASP page which is only ever called with a querystring
Nothing has ever indexed this page, I guess because the search engine spiders detected the ? and
ignored it. BUT, the lycos spider has been poking it's nose in and has been requesting the page.
Does this now mean that Lycos is indexing dynamically created pages? Anyone else notice if the other
search engines have started doing the same?