Bot Protection

Keitaro automatically detects bots by the following attributes:

  • IP in the bot database of Keitaro
  • IP in the user's list
  • UserAgent in the bot database of Keitaro
  • Part of UserAgent in the user's list
  • Spammers myip.ms in the black list (disabled by default)
  • Empty UserAgent

You can change the settings at the page «Maintenance > Settings > Bots».

How to protect a campaign from bots

You need to create a special stream in the campaign.

  1. Create a stream with the type «Forced» and add the filter «Bots» in the «IS» mode.
  2. Сhoose the necessary action. For example, «Show 404 not found».

How to cloak a website with Keitaro

Read page cloaking a website.

What types of bots are there in the bot database?

  • Bots of search engines: Google, Bing and others;
  • Bots of advertising services: Google Adwords, Facebook;
  • Bots of ad monitoring systems and disclosing of links.

How to update, and how often are the bot databases updated?

The bot databases are updated on the page «Maintenance > Geo DBs». Updates are issued 2 times a month.

How to add custom bot list?

At the page «Maintenance > Settings > Bots», press «Edit list» . You can write IPs and CIDR. Other formats are not supported.

Example:

78.12.31.1
92.12.1.0/24

Keitaro optimizes the list. It's okay when uploading 100.000, then you see 50.000 after saving.

How to add a large number of IPs to bot list?

1. Upload the file to the server. For example as file /var/www/keitaro/file.txt. 2. Run these commands

cd /var/www/keitaro
php bin/cli.php bots:import file.txt

What is the signature of bots?

It is a part of UserAgent. For example, for Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) the signature Googlebot.

How to add signature of bots

Open the page Settings > Bots. You can set the signatures separated by commas:

signature1, signature2, ...

How to detect bots through a provider

The provider database is not supplied with Keitaro due to limitations imposed by the IP2Location license agreement.

You can connect the database in two ways:

  1. Buy the license. Enter the license key at the page «Maintenance > Settings > Integration».
  2. Download the GeoISP.dat file manually to the directory /var/geoip/GeoISP/.

After integrating the database, you can use the «ISP» filter in the streams.