Date: Thu, 13 Oct 2005 00:22:34 -0500
From: John Goerzen <jgoerzen@complete.org>
To: gopher@complete.org
Subject: [gopher] Re: New Gopher Wayback Machine Bot
Message-ID: <20051013052234.GA30200@katherina.lan.complete.org>
References: <20051012180132.GA19083@complete.org>
 <200510122345.QAA17070@floodgap.com>
 <20051013020752.GA20333@SDF.LONESTAR.ORG>
 <20051013042826.GA29058@katherina.lan.complete.org>
MIME-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20051013042826.GA29058@katherina.lan.complete.org>
User-Agent: Mutt/1.5.11
Content-Transfer-Encoding: 8bit
Sender: gopher-bounce@complete.org
Errors-to: gopher-bounce@complete.org
Precedence: bulk
Reply-to: gopher@complete.org

On Wed, Oct 12, 2005 at 11:28:26PM -0500, John Goerzen wrote:
> On Wed, Oct 12, 2005 at 09:07:52PM -0500, Benn Newman wrote:
> I will be doing more work tomorrow to make it be more round-robin so
> as to not put much load on any one server at once.

And this bit is now done as well.  It will make one connection to
every server in its "to-spider" list before it ever connects to one
again.

I'll also keep an eye on it and make sure nobody gets pounded at the
very end.

-- John