How to Prevent Search Engine Bots to Meta Tag

To prevent all search bots from indexing a page:

<meta name="robots" content="noindex">

To prevent just Google:

<meta name="googlebot" content="noindex">

If you have control over it, you probably want to add nofollow to links that point to that page as well:

<a href="privatepage.html" rel="nofollow">Link to private page</a>

Theoretically that's not needed for Google, which claims to drop all pages with noindex from their directory. But there are more search engines out there and it doesn't hurt.
Reference URL

Also, you might want to add a robots.txt to prevent the bots from indexing your directories, it might look like this:

User-agent: *
Disallow: /

which would prevent all bots from indexing; of course you can also only block certain bots by their user agent; google it :-)
Previous
Next Post »