Received: with LISTAR (v1.0.0; list gopher);
 Sat, 13 Jan 2001 17:52:20 -0600 (CST)
Return-Path: <md@Linux.IT>
Delivered-To: gopher@complete.org
Received: from attila.bofh.it (attila.bofh.it [213.92.8.2])
	by pi.glockenspiel.complete.org (Postfix) with ESMTP id E18453B807
	for <gopher@complete.org>; Sat, 13 Jan 2001 17:52:19 -0600 (CST)
Received: by attila.bofh.it (Postfix, from userid 10)
	id 4B424600C8; Sun, 14 Jan 2001 00:49:01 +0100 (CET)
Received: by wonderland.linux.it (Postfix/Md, from userid 1001)
	id B9E3617E2A; Sun, 14 Jan 2001 00:21:28 +0100 (CET)
Date: Sun, 14 Jan 2001 00:21:28 +0100
From: Marco d'Itri <md@Linux.IT>
To: gopher@complete.org
Subject: [gopher] Re: Gopher "robots.txt" (was Re: New V-2 WAIS database)
Message-ID: <20010114002128.D4061@wonderland.linux.it>
References: <20010113201629.B2099@wonderland.linux.it>
 <200101132037.MAA12676@stockholm.ptloma.edu>
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.3.12i
In-Reply-To: <200101132037.MAA12676@stockholm.ptloma.edu>;
 from spectre@stockholm.ptloma.edu on Sat, Jan 13, 2001 at 12:37:17PM -0800
Content-Transfer-Encoding: 8bit
X-archive-position: 88
X-listar-version: Listar v1.0.0
Sender: gopher-bounce@complete.org
Errors-to: gopher-bounce@complete.org
X-original-sender: md@Linux.IT
Precedence: bulk
Reply-to: gopher@complete.org
X-list: gopher


On Jan 13, Cameron Kaiser <spectre@stockholm.ptloma.edu> wrote:

 >Good point. I am actually trying to think of a way like the HTTP robots.txt
 >that can more or less transparently tell V-2 what to stay out of. Suggestions?
I don't know well gopher yet, but you'd better find one before starting
to index my site, or your database will be filled with crap.

-- 
ciao,
Marco