If the alternative is to increase your instance size then it is at least worth looking at sitting behind Cloudflare, who are pretty effective at identifying the mass of unethical, Chinese AI scrapers you'll be facing...Recent activity on the Wiki has been traced to various bots (assumption is AI trawlers). Server loads are being looked at.
This has been a problem for at least a year. I wish that it could be masses of new players swarming to investigate the traveller wiki, but outside of Traveller circles I've yet to see any buzz.T5E is announced, Traveller wiki trawled by AI scrapers...
I wonder what could possibly be going on...
If the alternative is to increase your instance size then it is at least worth looking at sitting behind Cloudflare, who are pretty effective at identifying the mass of unethical, Chinese AI scrapers you'll be facing...
Wales has talked about the expanded cost for WikiMedia because of the crawlers not respecting the requests/code to stop scraping. I increased my donation to them because I want Wikipedia to remain strong.
Also, I am by no means an IT expert or anything, but Wikipedia is free for anyone to access or even edit without a login account, and they don't seem to have an issue (and I am sure they would be a prime target for AI-scrapers). And they operate by donation-support also. What do they use to avoid getting terminally bogged down by excessive AI and/or automated bots tasked by unscrupulous (ab)users?
Wales has talked about the expanded cost for WikiMedia because of the crawlers not respecting the requests/code to stop scraping. I increased my donation to them because I want Wikipedia to remain strong.
It's worth knowing that, unless something dranatic has changed this year, nothing that you contribute to Wikipedia in your funding drive goes to actually funding Wikipedia, which is already fully funded. It all goes on money that they contribute to other political causes.Wales has talked about the expanded cost for WikiMedia because of the crawlers not respecting the requests/code to stop scraping. I increased my donation to them because I want Wikipedia to remain strong.
Good luck to them prosecuting AI labs in Shenzhen and Hangzhou. The CCP doesn't tend to be very cooperative...They should find a way to totally deny them access if they refuse to respect requests/rules. If they find a way around after that they are hackers and should be prosecuted.
Good luck to them prosecuting AI labs in Shenzhen and Hangzhou. The CCP doesn't tend to be very cooperative...
It's worth knowing that, unless something dramatic has changed this year, nothing that you contribute to Wikipedia in your funding drive goes to actually funding Wikipedia, which is already fully funded. It all goes on money that they contribute to other political causes.
To be more precise, their internet hosting costs last year were $3.5M. Their awards and grants to other foundations and bodies amounted to $28.7M. Their expenses from attending conferences and other travel was twice the cost of hosting Wikipedia and all their other wikimedia projects together. But most of their costs go on their massive staff ($114M), only a minority of whom work on wikipedia itself.Thanks for sharing that. That is helpful to know.
Recent activity on the Wiki has been traced to various bots (assumption is AI trawlers). Server loads are being looked at.