Received: with ECARTIS (v1.0.0; list gopher); Wed, 23 Aug 2006 10:17:39 -0500 (CDT) Received: from netblock-66-159-214-137.dslextreme.com ([66.159.214.137] helo=floodgap.com ident=elvis) by glockenspiel.complete.org with esmtp (Exim 4.50) id 1GFuU1-0007kX-5y for gopher@complete.org; Wed, 23 Aug 2006 10:17:39 -0500 Received: (from spectre@localhost) by floodgap.com (6.6.6.666.1/2006.04.11) id k7NFIDGJ005586 for gopher@complete.org; Wed, 23 Aug 2006 08:18:13 -0700 From: Cameron Kaiser <spectre@floodgap.com> Message-Id: <200608231518.k7NFIDGJ005586@floodgap.com> Subject: [gopher] Re: Gopherspace archive In-Reply-To: <44EC663B.3020305@sdf.lonestar.org> from Benn Newman at "Aug 23, 6 09:29:15 am" To: gopher@complete.org Date: Wed, 23 Aug 2006 08:18:13 -0700 (PDT) X-Mailer: ELM [version 2.4ME+ PL39 (25)] MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8bit X-Spam-Status: No (score 0.2): AWL=0.120, FORGED_RCVD_HELO=0.05 X-Virus-Scanned: by Exiscan on glockenspiel.complete.org at Wed, 23 Aug 2006 10:17:39 -0500 X-archive-position: 1368 X-ecartis-version: Ecartis v1.0.0 Sender: gopher-bounce@complete.org Errors-to: gopher-bounce@complete.org X-original-sender: spectre@floodgap.com Precedence: bulk Reply-to: gopher@complete.org List-help: <mailto:ecartis@complete.org?Subject=help> List-unsubscribe: <mailto:gopher-request@complete.org?Subject=unsubscribe> List-software: Ecartis version 1.0.0 List-Id: Gopher <gopher.complete.org> X-List-ID: Gopher <gopher.complete.org> List-subscribe: <mailto:gopher-request@complete.org?Subject=subscribe> List-owner: <mailto:jgoerzen@complete.org> List-post: <mailto:gopher@complete.org> List-archive: <http://www.complete.org/mailinglists/archives/> X-list: gopher

> :)). With all the software and binary stuff taken out, I think it should > (nearly) manageable. The index file shouldn't be nearly as big as the > whole archive, I could then make a front end to that (yay for sed and awk!).

Looking at the size of V-2's index file, I wouldn't count on that if you have any kind of useful indexing. Yes, you won't be indexing binaries, but a large text file will have *plenty* of keywords, and an index of keywords itself has to be indexed by your database engine to be usefully searchable. That'll consume a platter or two.

I haven't decided yet if I can support hosting the archive. This sounds like a job for BitTorrent unless the other solution is to unpack the archive and let people take the pieces they want instead of downloading the whole thing. My uplink is a woeful 608k max ADSL line and the web and gopher servers both fight over it on a daily basis.

-- --------------------------------- personal: http://www.armory.com/~spectre/ --- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckaiser@floodgap.com -- Those are my principles. If you don't like them, I have others. -- G. Marx -