From: Cameron Kaiser <spectre@stockholm.ptloma.edu>
Message-Id: <200101150532.VAA09174@stockholm.ptloma.edu>
Subject: [gopher] Re: Gopher "robots.txt"
In-Reply-To: <20010114211300.A885@success-info.com> from emanuel at heatdeath
 organisation at "Jan 14, 1 09:13:00 pm"
To: gopher@complete.org
Date: Sun, 14 Jan 2001 21:32:45 -0800 (PST)
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
Sender: gopher-bounce@complete.org
Errors-to: gopher-bounce@complete.org
Precedence: bulk
Reply-to: gopher@complete.org


> It wouldn't be that much of a performance hit.  Rather than fetching the
> directory with a normal gopher request, then fetching the attributes for
> each entry of the directory, the robot could make a request something
> like this:
> 
> 	1/directoryF$+VIEWS+ABSTRACT+ROBOTS
> 
> which would fetch only the VIEWS, ABSTRACT, and ROBOTS attributes of
> every item in the directory (in addition to the INFO attribute, which
> contains the usual Gopher directory entry).  A fully Gopher+ supporting
> robot would need to get the VIEWS and ABSTRACT to build a complete index
> anyway.

This is a excellent idea, but I don't think we want to require all robots or
robot-like things to be gopher+ compliant. It would be nice, but I don't
think it should be mandatory.

Also, this doesn't solve the problem adequately for those *servers* which are
not gopher+ compliant, or certain subsets of indexed search servers that can't
or don't make gopher+ compliant responses (I can think of several immediately).

-- 
----------------------------- personal page: http://www.armory.com/~spectre/ --
 Cameron Kaiser, Point Loma Nazarene University * ckaiser@stockholm.ptloma.edu
-- Adore, v.: To venerate expectantly. -- Ambrose Bierce ----------------------