Basic Traffic Protection & Tools

[Fuckin Bill]

I'm going to try to come back and post more things in this thread as I find/remember/update them. If you want to trade traffic, you need to know how to protect and work the traffic to your advantage. Hopefully this will grow into a nice list of useful information and tools to keep your streets clean.

I'm going to keep this thread closed and sticky. If you have any comments or questions, feel free to start a new thread. This one will be used just to add new information.

To start things off, here's some mod-rewrite code for your .htaccess files that will block most known bad bots and site ripper programs that identify themselves to your web server.

RewriteEngine  On
RewriteCond %{HTTP_USER_AGENT} Ants [NC,OR]
RewriteCond %{HTTP_USER_AGENT} attach [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Backweb [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Bandit [NC,OR]
RewriteCond %{HTTP_USER_AGENT} BlackWidow [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Bot.mailto:craftbot@yahoo.com [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Buddy [NC,OR]
RewriteCond %{HTTP_USER_AGENT} CherryPicker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ChinaClaw [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Collector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Copier [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Crawler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Crescent [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Custo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} DA.4.0 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} DA.5.0 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} DA.5.3 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} DIIbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} DISCo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Download.Demon [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Download.Ninja [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Downloader [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Drip [NC,OR]
RewriteCond %{HTTP_USER_AGENT} eCatch [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EirGrabber [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EmailCollector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EmailSiphon [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EmailWolf [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Express.WebPictures [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ExtractorPro [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EyeNetIE [NC,OR]
RewriteCond %{HTTP_USER_AGENT} FileHound [NC,OR]
RewriteCond %{HTTP_USER_AGENT} FlashGet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} GetLeft [NC,OR]
RewriteCond %{HTTP_USER_AGENT} GetRight [NC,OR]
RewriteCond %{HTTP_USER_AGENT} GetWeb [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Gets [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Go.Zilla [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Go-Ahead-Got-It [NC,OR]
RewriteCond %{HTTP_USER_AGENT} gotit [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Grabber [NC,OR]
RewriteCond %{HTTP_USER_AGENT} GrabNet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Grafula [NC,OR]
RewriteCond %{HTTP_USER_AGENT} HMView [NC,OR]
RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ia_archiver [NC,OR]
RewriteCond %{HTTP_USER_AGENT} IBrowse [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Image.Stripper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Image.Sucker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Indy.Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} InterGET [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Internet.Ninja [NC,OR]
RewriteCond %{HTTP_USER_AGENT} JetCar [NC,OR]
RewriteCond %{HTTP_USER_AGENT} JOC [NC,OR]
RewriteCond %{HTTP_USER_AGENT} JustView [NC,OR]
RewriteCond %{HTTP_USER_AGENT} larbin [NC,OR]
RewriteCond %{HTTP_USER_AGENT} leech [NC,OR]
RewriteCond %{HTTP_USER_AGENT} LeechFTP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} lftp [NC,OR]
RewriteCond %{HTTP_USER_AGENT} likse [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Lickity [NC,OR]
RewriteCond %{HTTP_USER_AGENT} LinkWalker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Link.Sleuth [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Magnet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Mag-Net [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Mass.Downloader [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Memo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Microsoft.URL [NC,OR]
RewriteCond %{HTTP_USER_AGENT} MIDown.tool [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Mirror [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Mister.Pix [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Navroad [NC,OR]
RewriteCond %{HTTP_USER_AGENT} NearSite [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Net.Vampire [NC,OR]
RewriteCond %{HTTP_USER_AGENT} NetAnts [NC,OR]
RewriteCond %{HTTP_USER_AGENT} NetCaptor [NC,OR]
RewriteCond %{HTTP_USER_AGENT} NetSpider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} NetZIP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} NICErsPRO [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Octopus [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Offline [NC,OR]
RewriteCond %{HTTP_USER_AGENT} PageGrabber [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Papa.Foto [NC,OR]
RewriteCond %{HTTP_USER_AGENT} pcBrowser [NC,OR]
RewriteCond %{HTTP_USER_AGENT} PicaLoader [NC,OR]
RewriteCond %{HTTP_USER_AGENT} PictureRipper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} PicHunter [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Pockey [NC,OR]
RewriteCond %{HTTP_USER_AGENT} psbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Pump [NC,OR]
RewriteCond %{HTTP_USER_AGENT} RealDownload [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Reaper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Recorder [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ReGet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Siphon [NC,OR]
RewriteCond %{HTTP_USER_AGENT} sitecheck.internetseer.com [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SiteSnagger [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SmartDownload [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Snagger [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Snake [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SpaceBison [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Stripper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Sucker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SuperBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SuperHTTP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Surfbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} tAkeOut [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Teleport [NC,OR]
RewriteCond %{HTTP_USER_AGENT} IUPUI [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Vacuum [NC,OR]
RewriteCond %{HTTP_USER_AGENT} VoidEYE [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Web.Bandit [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Web.Image.Collector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Web.Sucker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebAuto [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebCopier [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Web.Downloader [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebEMailExtrac [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebGo.IS [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebFetch [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebLeacher [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebReaper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebRipper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebSauger [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Website.eXtractor [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Website.Quester [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Webster [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebStripper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebWhacker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebZIP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Whacker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Widow [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WWWOFFLE [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Xaldon.WebSpider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Zeus [NC] 
RewriteRule .*$ - [F]

This list has been checked and tested using http://www.wannabrowser.com/index.php

The list was compiled and tweaked from my own list of bots, and another that was posted recently by GTP Dave.

It is recommended that you check your own site with WannaBrowser as well to be sure it is functioning. I've seen several of these types of lists floating around that simply do nothing at all because coding is incorrect.

There are also a few that block the "wget" user agent. Doing this can be dangerous since many linux based scripts (Like just about all TGP scripts) use wget to download remote files. If you block wget from your gallery server, TGP's won't be able to grab your pics for thumbnailing, for example.

[Fuckin Bill]

Following up on the bot blocking, here's a slightly more advanced and more efficient set of rules for blocking bots. Big thanks to Raymor for the suggestions and comments that were used to make this version.

The following code will first check to see if the user is coming in on Internet Explorer, or one of the Mozilla/Gecko derived browsers (like Firefox). If it's one of these browsers, the server allows it and skips the checking of all the bad user agents. This will save considerable processing on large multi-file sites like TGP's.

If the browser is not IE/Mozilla, the server will check to see if it is a bad user agent. If it is bad, it gets a forbidden message. If it is still not matched to anything, like a cell or PDA device, it will be allowed in.

The end result is that common browsers are allowed in with minimal checks, unknown browsers are allowed through so you don't block new browsers/devices by default, and the bad bots are still kept away from your goodies.

## Check For Mozilla/MSIE And Allow It Before Checking All
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/[0-9]\.[0-9].*Gecko [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/[0-9]\.[0-9]\ \(compatible\;\ MSIE
RewriteRule ^(.*) - [S=1]

## If Not Mozilla/MSIE check for bad bot
RewriteCond %{HTTP_USER_AGENT} Ants [NC,OR]
RewriteCond %{HTTP_USER_AGENT} attach [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Backweb [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Bandit [NC,OR]
RewriteCond %{HTTP_USER_AGENT} BlackWidow [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Bot.mailto.craftbot.yahoo.com [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Buddy [NC,OR]
RewriteCond %{HTTP_USER_AGENT} CherryPicker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ChinaClaw [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Collector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Copier [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Crawler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Crescent [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Custo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} DA.4.0 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} DA.5.0 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} DA.5.3 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} DIIbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} DISCo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Download.Demon [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Download.Ninja [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Downloader [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Drip [NC,OR]
RewriteCond %{HTTP_USER_AGENT} eCatch [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EirGrabber [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EmailCollector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EmailSiphon [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EmailWolf [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Express.WebPictures [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ExtractorPro [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EyeNetIE [NC,OR]
RewriteCond %{HTTP_USER_AGENT} FileHound [NC,OR]
RewriteCond %{HTTP_USER_AGENT} FlashGet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} GetLeft [NC,OR]
RewriteCond %{HTTP_USER_AGENT} GetRight [NC,OR]
RewriteCond %{HTTP_USER_AGENT} GetWeb [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Gets [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Go.Zilla [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Go-Ahead-Got-It [NC,OR]
RewriteCond %{HTTP_USER_AGENT} gotit [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Grabber [NC,OR]
RewriteCond %{HTTP_USER_AGENT} GrabNet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Grafula [NC,OR]
RewriteCond %{HTTP_USER_AGENT} HMView [NC,OR]
RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ia_archiver [NC,OR]
RewriteCond %{HTTP_USER_AGENT} IBrowse [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Image.Stripper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Image.Sucker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Indy.Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} InterGET [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Internet.Ninja [NC,OR]
RewriteCond %{HTTP_USER_AGENT} JetCar [NC,OR]
RewriteCond %{HTTP_USER_AGENT} JOC [NC,OR]
RewriteCond %{HTTP_USER_AGENT} JustView [NC,OR]
RewriteCond %{HTTP_USER_AGENT} larbin [NC,OR]
RewriteCond %{HTTP_USER_AGENT} leech [NC,OR]
RewriteCond %{HTTP_USER_AGENT} LeechFTP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} lftp [NC,OR]
RewriteCond %{HTTP_USER_AGENT} likse [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Lickity [NC,OR]
RewriteCond %{HTTP_USER_AGENT} LinkWalker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Link.Sleuth [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Magnet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Mag-Net [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Mass.Downloader [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Memo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Microsoft.URL [NC,OR]
RewriteCond %{HTTP_USER_AGENT} MIDown.tool [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Mirror [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Mister.Pix [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Navroad [NC,OR]
RewriteCond %{HTTP_USER_AGENT} NearSite [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Net.Vampire [NC,OR]
RewriteCond %{HTTP_USER_AGENT} NetAnts [NC,OR]
RewriteCond %{HTTP_USER_AGENT} NetCaptor [NC,OR]
RewriteCond %{HTTP_USER_AGENT} NetSpider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} NetZIP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} NICErsPRO [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Octopus [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Offline [NC,OR]
RewriteCond %{HTTP_USER_AGENT} PageGrabber [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Papa.Foto [NC,OR]
RewriteCond %{HTTP_USER_AGENT} pcBrowser [NC,OR]
RewriteCond %{HTTP_USER_AGENT} PicaLoader [NC,OR]
RewriteCond %{HTTP_USER_AGENT} PictureRipper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} PicHunter [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Pockey [NC,OR]
RewriteCond %{HTTP_USER_AGENT} psbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Pump [NC,OR]
RewriteCond %{HTTP_USER_AGENT} RealDownload [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Reaper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Recorder [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ReGet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Siphon [NC,OR]
RewriteCond %{HTTP_USER_AGENT} sitecheck.internetseer.com [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SiteSnagger [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SmartDownload [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Snagger [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Snake [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SpaceBison [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Stripper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Sucker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SuperBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SuperHTTP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Surfbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} tAkeOut [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Teleport [NC,OR]
RewriteCond %{HTTP_USER_AGENT} IUPUI [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Vacuum [NC,OR]
RewriteCond %{HTTP_USER_AGENT} VoidEYE [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Web.Bandit [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Web.Image.Collector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Web.Sucker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebAuto [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebCopier [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Web.Downloader [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebEMailExtrac [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebGo.IS [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebFetch [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebLeacher [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebReaper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebRipper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebSauger [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Website.eXtractor [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Website.Quester [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Webster [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebStripper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebWhacker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebZIP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Whacker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Widow [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WWWOFFLE [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Xaldon.WebSpider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Zeus [NC] 
RewriteRule .*$ - [F,L]

Shortly I will try to get up the basics for protecting yourself from bad domains, such as image hotlinkers and hitbots. And then a detailed and (hopefully) easy to follow explanation of exactly what all this code does, how to make it all work together without problems, and most importantly, how to update it as you come across new threats to your sites and traffic.

[Fuckin Bill]

The following rewrite rules will allow you redirect of forbid traffic coming from a refferer you don't like. This can help with deflecting hitbot traffic, or traffic from sites that install trojans and exploits, so it doesn't skew your traffic statistics.

Keep in mind that if you use this on your sites, you should NOT redirect the traffic to a "pop-up hell" or some other trap. You will most likely be deflecting a lot of hitbots, but there will also be some real surfers in there who just got unlucky and came to you from a scumbag site. Don't abuse them more. If you're not going to just forbid the traffic, at least send them to a virus scanner or spybot page. You can actually help some people, and very possibly make a few extra dollars off them.

To redirect traffic to URL of your choice use this:

## Redirect Bad Domains
RewriteCond %{HTTP_REFERER} ^http://(www\.)?holyvoyeur\.com?.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?bbwlands\.com?.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?easytoons\.net?.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?easygay\.net?.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?teensoncock\.net?.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?bestworldtop\.com?.*$ [NC]
RewriteRule ^.*$ http://www.REDIRECT-URL-HERE.com  [R,L]

To give a forbidden error instead, change the last line to this:

RewriteRule ^.*$ -  [F,L]

To add more, simply copy one of the entries already there, paste it back in, and edit the name. Be sure to watch the last line when editing. All lines have an [OR] in them EXCEPT the last one. Placing an OR statement there will break it.

The redirect/forbid will match the domain names with or without the WWW. prefix, and does not worry about letter case.

This will stop or redirect the bad traffic before any of your scripts see it, keeping your stats from becoming falsely inflated by bad or bot traffic.

[Fuckin Bill]

If you're not sure what "hotlinking" is...It's the practice of using images, videos, or other large files hosted on a foreign web site as if they were part of your web pages.

If you run any type of adult site that contains images, you will learn why hotlinking is bad, and costly, very quickly. It's absolutely imperative that you have something in place to keep shady webmasters and surfers from using your images to promote their own goals.

The basic idea is to block ALL hotlinking by default, and then add specific sites which are allowed. It's easier to make a small list of allowed sites than it is to constantly add to a large list of disallowed sites. And this way you're protected by default, instead of waiting to be robbed.

The following code added into your .htaccess file after (or before) your other rewrite rules will block all hotlinking of jpg images, except for sites you specify:

## Allow image hotlinks from my domains
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?amateurpornspy\.com?.*$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?hardcorepornspy\.com?.*$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?thumbhustler\.com?.*$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?milfuniverse\.com?.*$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?thatpornpage\.com?.*$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?netpond\.com?.*$
RewriteRule .*\.jpg$      -        [F,L]

You can see above I have a small list of some of my sites, and Netpond, so I can hotlink my own images in my posts (or my sig) if I want to. You can also set things up to block GIF and other types of images, but 95% of adult site images are jpg. If we don't make it check for gifs and other more obscure file types, the server has less to process.

If you're starting to put some of these things together into your own .htaccess, it can make things a lot easier if you keep a "master" file already set up, and simply add each of your new domains to the hotlink section as you build them. Then you end up with 1 htaccess file that you can upload to any of your sites. You'll be able to use all your images on any of your domains that you want, but they will all be protected right from the start without having to rewrite the file every time.

It should also be noted that these rules will only work for NON-VIDEO files. Because of the way video player software works, hotlinking of videos can not be effectively blocked without the use of more sophisticated scripts and rewrite rules.