Welcome to ftp.nluug.nl Current directory: /ftp/os/Linux/distr/salix/x86_64/extra-15.0/source/perl/perl-www-robotrules/

Current bandwidth utilization 1365.92 Mbit/s

This module parses /robots.txt files as specified in "A Standard for
Robot Exclusion", at <http://www.robotstxt.org/wc/norobots.html>
Webmasters can use the /robots.txt file to forbid conforming robots
from accessing parts of their web site.
The parsed files are kept in a WWW::RobotRules object, and this
object provides methods to check if access to a given URL is
prohibited. The same WWW::RobotRules object can be used for one
or more parsed /robots.txt files on any number of hosts.

 Name                                                     Last modified      Size  
 Parent Directory                                                              -   
 README                                                   27-May-2022 12:45  488   
 WWW-RobotRules-6.02.tar.gz                               24-Aug-2016 02:43  8.8K  
 perl-www-robotrules.SlackBuild                           11-Mar-2022 06:34  3.1K  
 perl-www-robotrules.info                                 27-May-2022 12:45  321   
 slack-desc                                               27-May-2022 12:45  1.0K  

NLUUG - Open Systems. Open Standards
Become a member and get discounts on conferences and more, see the NLUUG website!