From Dev.to
I checked my nginx log for AI crawlers. A third of the “Googlebots” were fake
I went through two weeks of my site's nginx access log and counted every bot that said it was an AI crawler or a search engine. I only kept requests whose IP address really belongs to the company behi...