Fwd: [IMC-Tech] People to put in your block list...

[Daniel S. Lewart]

Paul, et al,

> >    Hi guys, I just noticed that there's a company with a very aggressive
> > spider that seems to be scanning activist web sites.  It scans out my
> > entire site every day, generating a few thousand hits every day, and
> > specifically ignoring robots.txt, and doing a lot of dumb things like
> > trying to log in and post a comment to every story.
> >    The spider is at 209.73.228.163 and gives "Webclipping.com" as it's
> > User Agent.  "Webclipping.com" advertises a service to help corporations
> > find and correct 'false information' on the web.  Punching
> > "209.73.228.163" into google reveals server logs that reveal these guys
> > are scanning everything in sight.

> Thoughts on this?

Yes--they should be banned ... and now they are, from:
	imc.cu.groogroo.com
	print.ucimc.org
	urbana.indymedia.org
	www.print.ucimc.org

Zach, you may ban them from the rest of your virtual hosts
or packet filter them.

Thanks!
Dan
-------------------------------------------------------------------------------
$ grep -c Webclipping /var/log/apache/imc-access.log
5074
-------------------------------------------------------------------------------
$ diff /etc/apache/httpd.conf.011218 /etc/apache/httpd.conf
31c31
< # LoadModule access_module /usr/lib/apache/1.3/mod_access.so
---
> LoadModule access_module /usr/lib/apache/1.3/mod_access.so
329a330,331
>	# Deny from webclipping.com
>	Deny from 209.73.228.163
355a358,359
>   # Deny from webclipping.com
>   Deny from 209.73.228.163
-------------------------------------------------------------------------------