Received: with ECARTIS (v1.0.0; list gopher);
 Wed, 12 Oct 2005 13:51:41 -0500 (CDT)
Received: from jgoerzen by glockenspiel.complete.org with local
	(Exim 4.50)
	id 1EPlhV-0005TC-8i; Wed, 12 Oct 2005 13:51:41 -0500
Date: Wed, 12 Oct 2005 13:51:41 -0500
From: John Goerzen <jgoerzen@complete.org>
To: gopher@complete.org
Subject: [gopher] Re: New Gopher Wayback Machine Bot
Message-ID: <20051012185141.GA21016@complete.org>
References: <20051012180132.GA19083@complete.org>
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20051012180132.GA19083@complete.org>
User-Agent: Mutt/1.5.9i
Content-Transfer-Encoding: 8bit
X-archive-position: 1108
X-ecartis-version: Ecartis v1.0.0
Sender: gopher-bounce@complete.org
Errors-to: gopher-bounce@complete.org
X-original-sender: jgoerzen@complete.org
Precedence: bulk
Reply-to: gopher@complete.org
List-help: <mailto:ecartis@complete.org?Subject=help>
List-unsubscribe: <mailto:gopher-request@complete.org?Subject=unsubscribe>
List-software: Ecartis version 1.0.0
List-Id: Gopher <gopher.complete.org>
X-List-ID: Gopher <gopher.complete.org>
List-subscribe: <mailto:gopher-request@complete.org?Subject=subscribe>
List-owner: <mailto:jgoerzen@complete.org>
List-post: <mailto:gopher@complete.org>
List-archive: <http://www.complete.org/mailinglists/archives/>
X-list: gopher

On Wed, Oct 12, 2005 at 01:01:32PM -0500, John Goerzen wrote:
> This bot is now running on my laptop and is spidering away.  The IP that
> you see connections from will vary depending on where my laptop is at a
> given time ;-)

Speaking of this, do we have any equivolent of robots.txt?  Cameron, do
you have some sort of exclude list or anything?

-- 
John Goerzen
Author, Foundations of Python Network Programming
http://www.amazon.com/exec/obidos/tg/detail/-/1590593715