From: Cameron Kaiser <spectre@floodgap.com>
Message-Id: <200405280245.TAA06966@floodgap.com>
Subject: [gopher] Re: Cicada Incomplete Gopher Census
In-Reply-To: <20040528022333.GA7147@nerds.cs.umd.edu> from Tim Fraser at "May
 27, 4 10:23:33 pm"
To: gopher@complete.org
Date: Thu, 27 May 2004 19:45:41 -0700 (PDT)
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
Sender: gopher-bounce@complete.org
Errors-to: gopher-bounce@complete.org
Precedence: bulk
Reply-to: gopher@complete.org

> I recently indulged my long-time fascination with Gopherspace by
> writing a spider program to determine how many Gophers are actually
> out there.  Of course, now that I'm finished, I've found Cameron
> Kaiser's post in the gopher mailing list archive announcing that he'd
> finished updating the the much nicer V-2 database at floodgap.org,
> even as my little toy spider was chugging away.  Seems I should have
> picked a different project to pass the time! ;^)

Actually, you can see the Floodgap census here (it's floodgap.com):

	gopher://gopher.floogap.com/1/world

After the V-2 cleanup this weekend, it has pared itself down to 255 unique
hosts and a database of about 1.8 million selectors. The last Veronica-2
database had 7.2 million selectors in the old sf-wais matrix, which was a
number probably inflated by filesystem loops and dead systems, so I
estimate gopherspace to be around 6 million selectors or so at present.
The robot is down for some tuning and updates, but I intend to start it up
again in a couple of days.

[yoink...]
> Near the end of my spidering, I realized that my
> primitive spider had been automatically banned from at least one site
> (floodgap.com) for behaving rudely in its quest to traverse all of the
> site's directories looking for links to other sites.  Horrified that
> my little project to celebrate Gopher's longevity might be causing
> trouble for Gopher administrators, I cut the spidering of these last
> two sites short.  My apologies to anyone I might have inconvenienced!

It was? I don't remember blocking any IP addresses ... *puzzled*

-- 
---------------------------------- personal: http://www.armory.com/~spectre/ --
 Cameron Kaiser, Floodgap Systems Ltd * So. Calif., USA * ckaiser@floodgap.com
-- "I'd love to go out with you, but I have to floss my parakeet." ------------