Received: with ECARTIS (v1.0.0; list gopher); Wed, 12 Oct 2005 13:01:33 -0500 (CDT) Received: from jgoerzen by glockenspiel.complete.org with local (Exim 4.50) id 1EPkuy-0004yZ-HK; Wed, 12 Oct 2005 13:01:32 -0500 Date: Wed, 12 Oct 2005 13:01:32 -0500 From: John Goerzen <jgoerzen@complete.org> To: gopher@complete.org Subject: [gopher] New Gopher Wayback Machine Bot Message-ID: <20051012180132.GA19083@complete.org> Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.9i Content-Transfer-Encoding: 8bit X-archive-position: 1107 X-ecartis-version: Ecartis v1.0.0 Sender: gopher-bounce@complete.org Errors-to: gopher-bounce@complete.org X-original-sender: jgoerzen@complete.org Precedence: bulk Reply-to: gopher@complete.org List-help: <mailto:ecartis@complete.org?Subject=help> List-unsubscribe: <mailto:gopher-request@complete.org?Subject=unsubscribe> List-software: Ecartis version 1.0.0 List-Id: Gopher <gopher.complete.org> X-List-ID: Gopher <gopher.complete.org> List-subscribe: <mailto:gopher-request@complete.org?Subject=subscribe> List-owner: <mailto:jgoerzen@complete.org> List-post: <mailto:gopher@complete.org> List-archive: <http://www.complete.org/mailinglists/archives/> X-list: gopher

Hello,

Since Gopher servers are dropping off the 'net at an unfortunate rate, I have decided to write a bot to try to index all gopherspace that is still reachable.

This bot is now running on my laptop and is spidering away. The IP that you see connections from will vary depending on where my laptop is at a given time ;-)

I have set it to make no more than 1 simultaneous connection to any given host. Hopefully that should make it nice enough to not cause anyone any trouble.

Cameron, floodgap.com seems to have some sort of rate limiting and keeps giving me a Connection refused error after a certain number of documents have been spidered.

-- John