intertwingly

It’s just data

robots.txt vs referrer spam


Many people have been doing this for a while, but as a public service announcement for those who may not have thought to do this:

A site can set up a robots.txt file which tells robots (which includes Google) to not index a particular file or directory.

Here's mine.  It is pretty easy to do.

Will it stop referrer spam?  No.  Many spammers are quite satisfied with fractional percent return on investment.  But it allows you to opt out.