@spla Yeah. I don't actually mind being scraped for a search engine, since they respect robots.txt and are slow. But these new AI data scrapers have no manners and no regard for your server. Imagine the energy usage!
N
nicd@masto.ahlcode.fi
@nicd@masto.ahlcode.fi
Posts
-
So here's what took down my Forgejo instance today. -
So here's what took down my Forgejo instance today.@spla Could it have been scraping too, or was it actively malicious (not that there's much difference I guess)? I do also fear that at some point I'll run into some DoSing dweeb, but for now I've been able to run my server in peace. Except for this AI crap.
It sucks how asymmetric the balance of power is here. I just want to be left alone and run my little server to host my own projects.
-
So here's what took down my Forgejo instance today.So here's what took down my Forgejo instance today. Alibaba ran a massive scraping operation on my poor tiny instance, probably to train a shitty AI. It also scraped the archive endpoints, generating zip archives of all the commits in all the repos until my disk ran out. Taught me to turn that feature off at least.
Yes, I have robots.txt. They don't respect it. I had to block their IP range, but no doubt they'll be back with another. 60 rps, wtf?!