Re: Excite IssuesFrom Marge on 14 Jan '99
replying to Excite Issues posted by GDS
I got dropped from Excite about three months ago, and have turned
handsprings trying to get relisted. Finally broke down and wrote to
their tech support. Come to find out, at one time (about three months
ago!) my ISP was mucking about in my site. They planted an EMPTY
robots.txt file there. I left it, thinking that 'cause it was empty,
the robots would blythely visit.
WRONG. According to Excite tech support, the presence of an empty
Robots.txt file can be enough to keep them from visiting and spidering
so if you are getting totally neglected by a spider that won't even
answer an invitation...that might be something to check for.