Massive Image scrapping from Liveinternet.ru since moving to Miva Merchant

[tammu]

We moved our site from a Windows server where our site was built in ASP.NET and C# to a Linux Server with proper firewalls. But we are getting a ton of hits from users of LiveInternet.ru.

Any way to block these or reduce the hits.

Thanks and Appreciate it

[ILoveHostasaurus]

Re: Massive Image scrapping from Liveinternet.ru since moving to Miva Merchant

Only two explanations I'd have for something like that occurring would be 1) perhaps the html code output by Merchant is better understood by whatever that bot is and it was getting stuck on your old site unable to parse something to be able to crawl it. Or 2) perhaps the old host had Russia blocked or something along those lines.

Merchant doesn't perform any task that would solicit such crawling but of course the development crew spends a lot of time on producing code that is hopefully crawled well by bots (preferably goods ones).

If you wanted to block image requests against your site that don't have a referrer of your site, you could use this code in a .htaccess file in your images directory:

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^(http|https)://(www.)?mydomain.com.*$ [NC]
RewriteRule .(gif|jpg)$ - [F]

It will basically just block image requests if the referring page is not one on your domain. This would of course preclude you from hot linking your images into things like Ebay, Amazon, etc. if that is something you do.