From: Cameron Kaiser <spectre@floodgap.com>
Message-Id: <200608231518.k7NFIDGJ005586@floodgap.com>
Subject: [gopher] Re: Gopherspace archive
In-Reply-To: <44EC663B.3020305@sdf.lonestar.org> from Benn Newman at "Aug 23,
 6 09:29:15 am"
To: gopher@complete.org
Date: Wed, 23 Aug 2006 08:18:13 -0700 (PDT)
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
Sender: gopher-bounce@complete.org
Errors-to: gopher-bounce@complete.org
Precedence: bulk
Reply-to: gopher@complete.org

> :)). With all the software and binary stuff taken out, I think it should 
> (nearly) manageable. The index file shouldn't be nearly as big as the 
> whole archive, I could then make a front end to that (yay for sed and awk!).

Looking at the size of V-2's index file, I wouldn't count on that if you have
any kind of useful indexing. Yes, you won't be indexing binaries, but a large
text file will have *plenty* of keywords, and an index of keywords itself has
to be indexed by your database engine to be usefully searchable. That'll
consume a platter or two.

I haven't decided yet if I can support hosting the archive. This sounds like
a job for BitTorrent unless the other solution is to unpack the archive and
let people take the pieces they want instead of downloading the whole thing.
My uplink is a woeful 608k max ADSL line and the web and gopher servers both
fight over it on a daily basis.

-- 
--------------------------------- personal: http://www.armory.com/~spectre/ ---
  Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckaiser@floodgap.com
-- Those are my principles. If you don't like them, I have others. -- G. Marx -