Date: Sun, 14 Jan 2001 15:25:37 -0800
From: emanuel at heatdeath organisation <emanuel@heatdeath.org>
To: gopher@complete.org
Subject: [gopher] Re: Gopher "robots.txt"
Message-ID: <20010114152537.B1974@success-info.com>
Mail-Followup-To: emanuel at heatdeath organisation <emanuel@heatdeath.org>,
	gopher@complete.org
References: <20010113193659.A20066@mothra>
 <200101140056.QAA12702@stockholm.ptloma.edu> <20010114172831.A25812@mothra>
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5i
In-Reply-To: <20010114172831.A25812@mothra>;
 from s2mdalle@titan.vcu.edu on Sun, Jan 14, 2001 at 05:28:31PM -0500
Content-Transfer-Encoding: 8bit
Sender: gopher-bounce@complete.org
Errors-to: gopher-bounce@complete.org
Precedence: bulk
Reply-to: gopher@complete.org


On Sun, Jan 14, 2001 at 05:28:31PM -0500, David Allen wrote:
> That's not a bad idea...I'd prefer personally to put whatever
> information that this null statement has in the gopher+ fields to
> avoid overloading the meaning of the other fields (type, name, host,
> port).  So maybe something like this to disallow the debian
> subdirectory:
> 
> iFFerror.hostF909FDisallow: selector_for_debian_directory

Why not put this information in the Gopher+ attribute information?  Have
an attribute that means "ignore this selector" to the robot.  This way
clients that don't know about it will never see it.  If it's set for a
directory, the robot will ignore than directory and will never see
anything liked by the directory.

Isn't this kind of thing what the Gopher+ attribute information is for?

-- 
emanuel at heatdeath organisation
gopher.heatdeath.org