Vulnerability Assessment & Network Security Forums

If through a vulnerability assessment, a network security issue is detected for the vulnerability below, applying the appropriate security patches in a timely matter is very important. If you have detected that your system has already been compromised, following CERT's Network Security recovery document will assist with recommended steps for system recovery.

Home >> Browse Vulnerability Assessment Database >> CGI abuses >> Robots.txt Information Disclosure

Vulnerability Assessment Details

Robots.txt Information Disclosure

Vulnerability Assessment Summary
Checks for a web server's robots.txt

Detailed Explanation for this Vulnerability Assessment

Summary :

The remote web server contains a 'robots.txt' file.

Description :

The remote host contains a file named 'robots.txt' that is intended to
prevent web 'robots' from visiting certain directories in a web site for
maintenance or indexing purposes. A malicious user may also be able to
use the contents of this file to learn of sensitive documents or
directories on the affected site and either retrieve them directly or
target them for other attacks.

See also :

http://www.robotstxt.org/wc/exclusion.html

Solution :

Review the contents of the site's robots.txt file, use Robots META tags
instead of entries in the robots.txt file, and/or adjust the web
server's access controls to limit access to sensitive material.

Network Security Threat Level:

None

Networks Security ID:

Vulnerability Assessment Copyright: This script is Copyright (C) 2005-2006 Tenable Network Security

Cables, Connectors