Hello and welcome to our community! Is this your first visit?
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 1 of 1
  1. #1
    New Coder
    Join Date
    Nov 2008
    Thanked 0 Times in 0 Posts

    No robots in session secured area... please

    So I have an area secured by sessions where people login, nothing within this area can be used unless you have have this session.

    I have only recently started to become concerned about robots accessing this area - whats the advise on this?

    The area is accessed via a post form so login variables are not sent with the url, I have the usual meta in all php files within this section:

    <meta name="googlebot" content="noarchive" />
    <meta name="googlebot" content="nosnippet" />
    <meta name="googlebot" content="noindex" />
    <meta name="googlebot" content="nofollow"/>
    and a:
    User-agent: *
    Disallow: /
    in my robots.txt

    Allthough I feel there maybe a way to do this with .htaccess

    Any advice would be very welcome on stoping any robots caching anything in this area, thanks in advance.

    Thinking about it can robots get in here without a password?

    So a bit more research I could use in my .htaccess:

       RewriteEngine on
       RewriteCond %{HTTP_USER_AGENT} ^bot
       RewriteRule ^(.*)$ http://www.robotstxt.org/
    or in my virtual hosts file:

      <Directory />
        BrowserMatch ^bot deny_host
        Deny from env=deny_host
    hmmm is there a better way to do this? (without involving placing and updating all bots in these files)
    Last edited by chidge; 01-29-2010 at 11:16 AM.


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts