Date: Sun, 14 Jan 2001 21:13:00 -0800
From: emanuel at heatdeath organisation <emanuel@heatdeath.org>
To: gopher@complete.org
Subject: [gopher] Re: Gopher "robots.txt"
Message-ID: <20010114211300.A885@success-info.com>
Mail-Followup-To: emanuel at heatdeath organisation <emanuel@heatdeath.org>,
	gopher@complete.org
References: <20010113193659.A20066@mothra>
 <200101140056.QAA12702@stockholm.ptloma.edu> <20010114172831.A25812@mothra>
 <20010114152537.B1974@success-info.com> <20010114223427.B26948@mothra>
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5i
In-Reply-To: <20010114223427.B26948@mothra>;
 from s2mdalle@titan.vcu.edu on Sun, Jan 14, 2001 at 10:34:27PM -0500
Content-Transfer-Encoding: 8bit
Sender: gopher-bounce@complete.org
Errors-to: gopher-bounce@complete.org
Precedence: bulk
Reply-to: gopher@complete.org


> I guess it depends on the robot.  It is sort of what the attribute
> information is for, but should it be the case that a robot has to
> fetch the attribute information for every single directory it ever
> fetches?  That could cause a hit on performance/bandwidth since a
> robot can't fetch just the resource and be safe with that.

It wouldn't be that much of a performance hit.  Rather than fetching the
directory with a normal gopher request, then fetching the attributes for
each entry of the directory, the robot could make a request something
like this:

	1/directoryF$+VIEWS+ABSTRACT+ROBOTS

which would fetch only the VIEWS, ABSTRACT, and ROBOTS attributes of
every item in the directory (in addition to the INFO attribute, which
contains the usual Gopher directory entry).  A fully Gopher+ supporting
robot would need to get the VIEWS and ABSTRACT to build a complete index
anyway.

-- 
emanuel at heatdeath organisation
gopher.heatdeath.org