D
@julian Thank you for your response and taking this seriously.
Please keep everyone informed. Feeding fediverse data to search engines (even accidentally, as this appears to be) is a breach of trust. How you handle this now is likely to be remembered by the fediverse for a long time.
@deadsuperhero It doesn't matter where the data is coming from, the effect is the same. Scraping done over AP is still scraping. The data (retrieved over AP in this case) is being republished without a "noindex" tag so it is being fed into search engines, including posts on your peertube server.
Hi @julian
I know you're very busy sitting on panels at #Fedicon and talking about how to make the fediverse better. Great.
Unfortunately you are still running a scraper that is feeding search engines.
You've been posting from the con (tip: we use alt text on pictures here on the fediverse), so I know you're online.
You're following me, so you'll have seen my question. @Gargron has spoken to you too I believe.
A day later, no acknowledgement or apology or fix or promise of a fix. Why?
@thisismissem @Gargron @jonny @onepict
That may be the case for some instance admins, but most users are not admins.
The bigger issue is that feeding fediverse toots into search engines violates conventions and the expectations of most users. That's what causes fedi-riots every time some bright spark does it.
@thisismissem @Gargron @jonny @onepict yes, where "best practice" == "if I don't want my instance defederated by the majority of the fediverse"
@thisismissem @Gargron @jonny @onepict
The problem, as Gargron identified, appears to be the lack of a "noindex" tag, which in Fediverse terms is like running an SMTP open relay - a misconfiguration rather than a fault in protocol - but which should not be the default in any software, and which will get you instablocked by the entire Internet.
@Gargron @jonny @onepict Thanks. Yes, I understand. I do hope it is merely an "oversight" as you put it.
Given the number of times we've had to slap down attempts to make the fediverse searchable it's astonishing that a fediverse developer wouldn't take more care. Mistake or not, it's a huge breach of trust.
@alex yes, it appears to be a forum that has recently patched in fediverse support without understanding or respecting our conventions.
@Gargron @jonny @onepict As far as I can tell all public and unlisted posts are being posted publicly on the web by nodebb and have been picked up by search engines.
I realise everything on here is effectively "public" including DMs, but there has been strong resistance until now from the community to making the fediverse searchable.
NodeBB has broken that expectation.
@Rairii @onepict @julian@community.nodebb.org @julian@fietkau.social Even public posts should not be showing up in search engines.
Hi @ordnung
Are you aware that posts are being scraped and reblogged by community.nodebb.org?
The #nodebb software is reposting content, including unlisted posts and effectively making the fediverse searchable.
This looks like a #Fediblock to me.
@julian@fietkau.social @julian@community.nodebb.org @onepict I have no problem with Julian following me. I do have a problem with his software reposting everything I write (including unlisted posts) on the public web and making that searchable by engines.
That is explicitly against what we allow on the Fediverse.
Julian - you need to fix your software and respect the conventions the community has established for the Fediverse.
@onepict It is forum software, but is scraping posts from the fediverse (incl. Mastodon and PeerTube) and reposting that on the public web for search engines to discover.
AIUI making the fediverse searchable is an explicit non-goal in the community, and every previous attempt to do so has been met by blocks and defederation.
Can anyone tell me what is #NodeBB and why is it scraping and republishing fediverse content without consent?
