Can anyone tell me what is #NodeBB and why is it scraping and republishing fediverse content without consent?
-
Hi @ordnung
Are you aware that posts are being scraped and reblogged by community.nodebb.org?
The #nodebb software is reposting content, including unlisted posts and effectively making the fediverse searchable.
This looks like a #Fediblock to me.
Forum software NodeBB joins the fediverse
This might have something to do with it.
I'm pretty sure a Fedi instance is supposed to cache posts. That is literally what one is supposed to do.
(So I searched for some of my previously deleted accounts. They didn't cache any of those. It seems they're being reasonably good Fedi citizens and respecting deletes.)
-
@Gargron @jonny @onepict As far as I can tell all public and unlisted posts are being posted publicly on the web by nodebb and have been picked up by search engines.
I realise everything on here is effectively "public" including DMs, but there has been strong resistance until now from the community to making the fediverse searchable.
NodeBB has broken that expectation.
-
Forum software NodeBB joins the fediverse
This might have something to do with it.
I'm pretty sure a Fedi instance is supposed to cache posts. That is literally what one is supposed to do.
(So I searched for some of my previously deleted accounts. They didn't cache any of those. It seems they're being reasonably good Fedi citizens and respecting deletes.)
@alex @dentangle @ordnung Well that explains why they just did it.
It's just another thing to connect
-
Forum software NodeBB joins the fediverse
This might have something to do with it.
I'm pretty sure a Fedi instance is supposed to cache posts. That is literally what one is supposed to do.
(So I searched for some of my previously deleted accounts. They didn't cache any of those. It seems they're being reasonably good Fedi citizens and respecting deletes.)
@alex yes, it appears to be a forum that has recently patched in fediverse support without understanding or respecting our conventions.
-
@dentangle @jonny @onepict As I said, that page should have a noindex tag on it (if you know what that is), and I consider it an oversight that it doesn't. I've let the NodeBB folks know about it a few minutes ago. However, the existence of this page is completely normal. The equivalent page on mastodon.social is mastodon.social/@dentangle@chaos.social, and it is how I can talk to you despite not having an account on chaos.social.
-
@dentangle @jonny @onepict As I said, that page should have a noindex tag on it (if you know what that is), and I consider it an oversight that it doesn't. I've let the NodeBB folks know about it a few minutes ago. However, the existence of this page is completely normal. The equivalent page on mastodon.social is mastodon.social/@dentangle@chaos.social, and it is how I can talk to you despite not having an account on chaos.social.
@Gargron @dentangle @jonny I'm aware of backfilling and profiles existing on fediverse instances. So are other folks.
My main issue is it being searchable on search engines. Plus mushing everything together without respecting the public/quiet public stuff.
Thank you Eugen for making them aware.
-
@dentangle @jonny @onepict As I said, that page should have a noindex tag on it (if you know what that is), and I consider it an oversight that it doesn't. I've let the NodeBB folks know about it a few minutes ago. However, the existence of this page is completely normal. The equivalent page on mastodon.social is mastodon.social/@dentangle@chaos.social, and it is how I can talk to you despite not having an account on chaos.social.
@Gargron @jonny @onepict Thanks. Yes, I understand. I do hope it is merely an "oversight" as you put it.
Given the number of times we've had to slap down attempts to make the fediverse searchable it's astonishing that a fediverse developer wouldn't take more care. Mistake or not, it's a huge breach of trust.
-
@Gargron @jonny @onepict Thanks. Yes, I understand. I do hope it is merely an "oversight" as you put it.
Given the number of times we've had to slap down attempts to make the fediverse searchable it's astonishing that a fediverse developer wouldn't take more care. Mistake or not, it's a huge breach of trust.
@dentangle @Gargron @jonny @onepict so at a protocol level "quiet public" doesn't really exist, all that happens in mastodon is that as:Public gets moved from `to`to `cc`, so they're effectively the same audience being addressed.
So NodeBB is actually right, at a protocol level, to treat public and "quiet public" as the same.
Though it sounds like steps will be taken to prevent indexing & display (when unauthenticated) of remote content outside of the context of a thread (you can't exactly mark sections of a page as noindex)
-
@dentangle @Gargron @jonny @onepict so at a protocol level "quiet public" doesn't really exist, all that happens in mastodon is that as:Public gets moved from `to`to `cc`, so they're effectively the same audience being addressed.
So NodeBB is actually right, at a protocol level, to treat public and "quiet public" as the same.
Though it sounds like steps will be taken to prevent indexing & display (when unauthenticated) of remote content outside of the context of a thread (you can't exactly mark sections of a page as noindex)
@thisismissem @Gargron @jonny @onepict
The problem, as Gargron identified, appears to be the lack of a "noindex" tag, which in Fediverse terms is like running an SMTP open relay - a misconfiguration rather than a fault in protocol - but which should not be the default in any software, and which will get you instablocked by the entire Internet.
-
@thisismissem @Gargron @jonny @onepict
The problem, as Gargron identified, appears to be the lack of a "noindex" tag, which in Fediverse terms is like running an SMTP open relay - a misconfiguration rather than a fault in protocol - but which should not be the default in any software, and which will get you instablocked by the entire Internet.
@dentangle @Gargron @jonny @onepict right, best practice is to not make remote content directly viewable without authentication (but it may still appear in thread/reply views without authentication)
-
@dentangle @Gargron @jonny @onepict right, best practice is to not make remote content directly viewable without authentication (but it may still appear in thread/reply views without authentication)
@thisismissem @Gargron @jonny @onepict yes, where "best practice" == "if I don't want my instance defederated by the majority of the fediverse"
-
@thisismissem @Gargron @jonny @onepict yes, where "best practice" == "if I don't want my instance defederated by the majority of the fediverse"
@dentangle @Gargron @jonny @onepict the source of that best practice is more around rehosting random content and consequently having liability for that content.
-
@dentangle @Gargron @jonny @onepict the source of that best practice is more around rehosting random content and consequently having liability for that content.
@thisismissem @Gargron @jonny @onepict
That may be the case for some instance admins, but most users are not admins.
The bigger issue is that feeding fediverse toots into search engines violates conventions and the expectations of most users. That's what causes fedi-riots every time some bright spark does it.
-
Can anyone tell me what is #NodeBB and why is it scraping and republishing fediverse content without consent?
Hi @julian
I know you're very busy sitting on panels at #Fedicon and talking about how to make the fediverse better. Great.
Unfortunately you are still running a scraper that is feeding search engines.
You've been posting from the con (tip: we use alt text on pictures here on the fediverse), so I know you're online.
You're following me, so you'll have seen my question. @Gargron has spoken to you too I believe.
A day later, no acknowledgement or apology or fix or promise of a fix. Why?
-
Hi @julian
I know you're very busy sitting on panels at #Fedicon and talking about how to make the fediverse better. Great.
Unfortunately you are still running a scraper that is feeding search engines.
You've been posting from the con (tip: we use alt text on pictures here on the fediverse), so I know you're online.
You're following me, so you'll have seen my question. @Gargron has spoken to you too I believe.
A day later, no acknowledgement or apology or fix or promise of a fix. Why?
Hi dentangle@chaos.social, I haven't been at a laptop this entire day since 7am this morning.
Around then I added a change to the link tags sent for remote profiles so that they point to the canonical source (your actual profile).
I'll likely just put in a redirect to your profile so it won't be accessible.
-
dentangle@chaos.social I appreciate your civility so far while I work through what needs to be done about this.
-
Can anyone tell me what is #NodeBB and why is it scraping and republishing fediverse content without consent?
@dentangle@chaos.social Quick question, what makes you think this is a scraper? NodeBB is forum software that implements ActivityPub and federates using the protocol.
-
@dentangle@chaos.social Quick question, what makes you think this is a scraper? NodeBB is forum software that implements ActivityPub and federates using the protocol.
@deadsuperhero It doesn't matter where the data is coming from, the effect is the same. Scraping done over AP is still scraping. The data (retrieved over AP in this case) is being republished without a "noindex" tag so it is being fed into search engines, including posts on your peertube server.
-
@julian Thank you for your response and taking this seriously.
Please keep everyone informed. Feeding fediverse data to search engines (even accidentally, as this appears to be) is a breach of trust. How you handle this now is likely to be remembered by the fediverse for a long time.
-
@julian Thank you for your response and taking this seriously.
Please keep everyone informed. Feeding fediverse data to search engines (even accidentally, as this appears to be) is a breach of trust. How you handle this now is likely to be remembered by the fediverse for a long time.
dentangle@chaos.social the
noindextag has been added to all remote profiles.
