bot invasion

Talk nerdy to us here.
Post Reply
User avatar
C-Dark
Reactions:
Posts: 2591
Joined: January 5th, 2012, 6:00 pm
Answers: 24
Location: Berlin (Germany)

bot invasion

Post by C-Dark » October 18th, 2018, 6:10 am

Recently I had the same problem with our typo3 website at work. For the htaccess file there is a fast way to forbid folders to those bots

User-agent: *
Disallow: /subdir1/
Disallow: /subdir2/
Disallow: /subdir3/

I think this works pretty good, so they better crawl the page content not the file folders ;-)
Claudy ;-) Bookworm, Cat Person, Computer Whiz, Couch Potato, Eco-Friendly, Friendly, Frugal, Genius, Nurturing, Social butterfly, Supernatural fan, Virtuoso in the past, Workaholic, Brooding, No Sense of Humor

User avatar
C-Dark
Reactions:
Posts: 2591
Joined: January 5th, 2012, 6:00 pm
Answers: 24
Location: Berlin (Germany)

bot invasion

Post by C-Dark » October 18th, 2018, 6:23 am

And if you might have also problems with mail bots which send you requests via contact form -
I had this issue too and tricked them with a safty mandatory word question, this they cannot fill correct, I experienced (instead math questions they do answer absolutely right lol).
Claudy ;-) Bookworm, Cat Person, Computer Whiz, Couch Potato, Eco-Friendly, Friendly, Frugal, Genius, Nurturing, Social butterfly, Supernatural fan, Virtuoso in the past, Workaholic, Brooding, No Sense of Humor

Post Reply