Page 1 of 1

bot invasion

Posted: October 18th, 2018, 6:10 am
by C-Dark
Recently I had the same problem with our typo3 website at work. For the htaccess file there is a fast way to forbid folders to those bots

User-agent: *
Disallow: /subdir1/
Disallow: /subdir2/
Disallow: /subdir3/

I think this works pretty good, so they better crawl the page content not the file folders ;-)

bot invasion

Posted: October 18th, 2018, 6:23 am
by C-Dark
And if you might have also problems with mail bots which send you requests via contact form -
I had this issue too and tricked them with a safty mandatory word question, this they cannot fill correct, I experienced (instead math questions they do answer absolutely right lol).