Received: with LISTAR (v1.0.0; list gopher); Sat, 13 Jan 2001 14:39:04 -0600 (CST) Return-Path: Delivered-To: gopher@complete.org Received: from stockholm.ptloma.edu (stockholm.ptloma.edu [199.106.86.50]) by pi.glockenspiel.complete.org (Postfix) with ESMTP id 7FC4C3B807 for ; Sat, 13 Jan 2001 14:39:02 -0600 (CST) Received: (from spectre@localhost) by stockholm.ptloma.edu (8.9.1/8.9.1) id MAA12676 for gopher@complete.org; Sat, 13 Jan 2001 12:37:17 -0800 From: Cameron Kaiser Message-Id: <200101132037.MAA12676@stockholm.ptloma.edu> Subject: [gopher] Gopher "robots.txt" (was Re: New V-2 WAIS database) In-Reply-To: <20010113201629.B2099@wonderland.linux.it> from Marco d'Itri at "Jan 13, 1 08:16:29 pm" To: gopher@complete.org Date: Sat, 13 Jan 2001 12:37:17 -0800 (PST) X-Mailer: ELM [version 2.4ME+ PL39 (25)] MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8bit X-archive-position: 87 X-listar-version: Listar v1.0.0 Sender: gopher-bounce@complete.org Errors-to: gopher-bounce@complete.org X-original-sender: spectre@stockholm.ptloma.edu Precedence: bulk Reply-to: gopher@complete.org X-list: gopher > >New indexing will resume Sunday evening. > How can I tell it to skip some parts of my server? > I don't want it to download the whole 19 GB debian archive, or run some > thousand times the NNTP gateway. Good point. I am actually trying to think of a way like the HTTP robots.txt that can more or less transparently tell V-2 what to stay out of. Suggestions? -- ----------------------------- personal page: http://www.armory.com/~spectre/ -- Cameron Kaiser, Point Loma Nazarene University * ckaiser@stockholm.ptloma.edu -- Die, v.: To stop sinning suddenly. -- Elbert Hubbard -----------------------