Re: [gopher] Strange implementations of "Missing file"
On May 16, 2012, at 17:50 , Damien Carol wrote:
> Thanks for the source code, I will try to use something similar.
> If your server is a finger service, I will remove this address of the crawler.
The right way is to just ignore robots.txt for ports 79. The gopher request (and crawling) should be carried on normally.
- Kim
_______________________________________________
Gopher-Project mailing list
Gopher-Project@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/gopher-project
Reply to: