Date: Wed, 12 Oct 2005 13:01:32 -0500
From: John Goerzen <jgoerzen@complete.org>
To: gopher@complete.org
Subject: [gopher] New Gopher Wayback Machine Bot
Message-ID: <20051012180132.GA19083@complete.org>
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.9i
Content-Transfer-Encoding: 8bit
Sender: gopher-bounce@complete.org
Errors-to: gopher-bounce@complete.org
Precedence: bulk
Reply-to: gopher@complete.org

Hello,

Since Gopher servers are dropping off the 'net at an unfortunate rate, I
have decided to write a bot to try to index all gopherspace that is
still reachable.

This bot is now running on my laptop and is spidering away.  The IP that
you see connections from will vary depending on where my laptop is at a
given time ;-)

I have set it to make no more than 1 simultaneous connection to any
given host.  Hopefully that should make it nice enough to not cause
anyone any trouble.

Cameron, floodgap.com seems to have some sort of rate limiting and keeps
giving me a Connection refused error after a certain number of documents
have been spidered.

-- John