[REMOVE] "(?:\b)facebookexternalhit(?:\b)" good_bot

[doxycomp]

Removal Request?

Please List the User-Agent string or Referrer to be added/removed

/globalblacklist.conf:BrowserMatchNoCase "(?:\b)facebookexternalhit(?:\b)" good_bot

Facebook is not a good bot and ignoring robots.txt and spamming the server with requests.

For Additions: Please include a log sample 3-5 lines is adequate

It defaults to good_bot in globalblacklist.conf and seemingly cannot be set as bad_bot in blacklist-user-agents.conf

Thank you!

[doxycomp]

Just in case, sombody else is struggling with this, for the moment we added some lines to the top of our .htaccess, to block the facebook/meta crawlers:

RewriteCond %{HTTP_USER_AGENT} ^facebookexternalhit.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^meta-externalagent.*
RewriteRule .* - [F,L]

Kind regards :)