Can anyone tell me what is #NodeBB and why is it scraping and republishing fediverse content without consent?
-
@dentangle I thought it was just forum software.
I wasn't aware that everything was being republished.
@onepict It is forum software, but is scraping posts from the fediverse (incl. Mastodon and PeerTube) and reposting that on the public web for search engines to discover.
AIUI making the fediverse searchable is an explicit non-goal in the community, and every previous attempt to do so has been met by blocks and defederation.
-
@dentangle NodeBB is a forum software that optionally supports ActivityPub, i.e. it's part of the fediverse (if the specific NodeBB instance wants). In my experience its lead ActivityPub dev @julian@community.nodebb.org (same first name, no relation
) is very approachable if you have questions about why and how it does things. He is currently at a convention though and may be a bit busy today.cc @onepict
@julian@fietkau.social @julian@community.nodebb.org @onepict I have no problem with Julian following me. I do have a problem with his software reposting everything I write (including unlisted posts) on the public web and making that searchable by engines.
That is explicitly against what we allow on the Fediverse.
Julian - you need to fix your software and respect the conventions the community has established for the Fediverse.
-
@onepict It is forum software, but is scraping posts from the fediverse (incl. Mastodon and PeerTube) and reposting that on the public web for search engines to discover.
AIUI making the fediverse searchable is an explicit non-goal in the community, and every previous attempt to do so has been met by blocks and defederation.
I must admit I'm a bit upset about everything being backfilled in and being uncategorised.
Especially since in Mastodon I explicitly selected to not be searchable on search engines.
It feels a little bit like the time Maven ingested everything.
Just because this time it's a forum over ActivityPub doesn't make me feel happy about it.
https://dotart.blog/cobbles/happy-bi-visibility-day -
@onepict It is forum software, but is scraping posts from the fediverse (incl. Mastodon and PeerTube) and reposting that on the public web for search engines to discover.
AIUI making the fediverse searchable is an explicit non-goal in the community, and every previous attempt to do so has been met by blocks and defederation.
Hi @ordnung
Are you aware that posts are being scraped and reblogged by community.nodebb.org?
The #nodebb software is reposting content, including unlisted posts and effectively making the fediverse searchable.
This looks like a #Fediblock to me.
-
I must admit I'm a bit upset about everything being backfilled in and being uncategorised.
Especially since in Mastodon I explicitly selected to not be searchable on search engines.
It feels a little bit like the time Maven ingested everything.
Just because this time it's a forum over ActivityPub doesn't make me feel happy about it.
https://dotart.blog/cobbles/happy-bi-visibility-day@onepict @dentangle secondary indexing that doesn't respect prefs is v bad, i have yet to figure out how in the world nodebb federates because it doesn't work like any other software that treats posts like author-owned entities, even lemmy handles this better.
-
@julian@fietkau.social @julian@community.nodebb.org @onepict I have no problem with Julian following me. I do have a problem with his software reposting everything I write (including unlisted posts) on the public web and making that searchable by engines.
That is explicitly against what we allow on the Fediverse.
Julian - you need to fix your software and respect the conventions the community has established for the Fediverse.
@dentangle @julian @julian @onepict indeed - https://docs.nodebb.org/activitypub/visibility/
"NodeBB does not differentiate between these separate states. Follower collections are ignored, and unlisted/quiet objects are considered fully public posts in NodeBB."
the first thing, looking at the code to try to confirm -> followers-only posts will just get discarded I think?
but yes, to nodebb, unlisted == public: https://github.com/NodeBB/NodeBB/blob/83a55f6adcd246920ba08415dcdf46505503c4a4/src/activitypub/inbox.js#L38-L42
(public address in cc means unlisted, this is treated as same as public) -
@dentangle @julian @julian @onepict indeed - https://docs.nodebb.org/activitypub/visibility/
"NodeBB does not differentiate between these separate states. Follower collections are ignored, and unlisted/quiet objects are considered fully public posts in NodeBB."
the first thing, looking at the code to try to confirm -> followers-only posts will just get discarded I think?
but yes, to nodebb, unlisted == public: https://github.com/NodeBB/NodeBB/blob/83a55f6adcd246920ba08415dcdf46505503c4a4/src/activitypub/inbox.js#L38-L42
(public address in cc means unlisted, this is treated as same as public)@Rairii @onepict @julian@community.nodebb.org @julian@fietkau.social Even public posts should not be showing up in search engines.
-
@Rairii @onepict @julian@community.nodebb.org @julian@fietkau.social Even public posts should not be showing up in search engines.
@dentangle @onepict @julian @julian i agree, but that's technically a different issue: currently there is no support for the discoverable/indexable flags. -
@onepict @dentangle secondary indexing that doesn't respect prefs is v bad, i have yet to figure out how in the world nodebb federates because it doesn't work like any other software that treats posts like author-owned entities, even lemmy handles this better.
@jonny @onepict @dentangle The copy of the profile should definitely not be served up in a way that can be indexed by search engines. This must be an oversight, I'll let them know. That being said, that kind of copy exists on every Mastodon server as well, the only difference is that NodeBB has a different look.
-
@jonny @onepict @dentangle The copy of the profile should definitely not be served up in a way that can be indexed by search engines. This must be an oversight, I'll let them know. That being said, that kind of copy exists on every Mastodon server as well, the only difference is that NodeBB has a different look.
@Gargron @jonny @onepict As far as I can tell all public and unlisted posts are being posted publicly on the web by nodebb and have been picked up by search engines.
I realise everything on here is effectively "public" including DMs, but there has been strong resistance until now from the community to making the fediverse searchable.
NodeBB has broken that expectation.
-
@Gargron @jonny @onepict As far as I can tell all public and unlisted posts are being posted publicly on the web by nodebb and have been picked up by search engines.
I realise everything on here is effectively "public" including DMs, but there has been strong resistance until now from the community to making the fediverse searchable.
NodeBB has broken that expectation.
^ @Taweret there is definitely something to nodebb's tendency to corndog the datasets
-
Hi @ordnung
Are you aware that posts are being scraped and reblogged by community.nodebb.org?
The #nodebb software is reposting content, including unlisted posts and effectively making the fediverse searchable.
This looks like a #Fediblock to me.
Forum software NodeBB joins the fediverse
This might have something to do with it.
I'm pretty sure a Fedi instance is supposed to cache posts. That is literally what one is supposed to do.
(So I searched for some of my previously deleted accounts. They didn't cache any of those. It seems they're being reasonably good Fedi citizens and respecting deletes.)
-
@Gargron @jonny @onepict As far as I can tell all public and unlisted posts are being posted publicly on the web by nodebb and have been picked up by search engines.
I realise everything on here is effectively "public" including DMs, but there has been strong resistance until now from the community to making the fediverse searchable.
NodeBB has broken that expectation.
-
Forum software NodeBB joins the fediverse
This might have something to do with it.
I'm pretty sure a Fedi instance is supposed to cache posts. That is literally what one is supposed to do.
(So I searched for some of my previously deleted accounts. They didn't cache any of those. It seems they're being reasonably good Fedi citizens and respecting deletes.)
@alex @dentangle @ordnung Well that explains why they just did it.
It's just another thing to connect
-
Forum software NodeBB joins the fediverse
This might have something to do with it.
I'm pretty sure a Fedi instance is supposed to cache posts. That is literally what one is supposed to do.
(So I searched for some of my previously deleted accounts. They didn't cache any of those. It seems they're being reasonably good Fedi citizens and respecting deletes.)
@alex yes, it appears to be a forum that has recently patched in fediverse support without understanding or respecting our conventions.
-
@dentangle @jonny @onepict As I said, that page should have a noindex tag on it (if you know what that is), and I consider it an oversight that it doesn't. I've let the NodeBB folks know about it a few minutes ago. However, the existence of this page is completely normal. The equivalent page on mastodon.social is mastodon.social/@dentangle@chaos.social, and it is how I can talk to you despite not having an account on chaos.social.
-
@dentangle @jonny @onepict As I said, that page should have a noindex tag on it (if you know what that is), and I consider it an oversight that it doesn't. I've let the NodeBB folks know about it a few minutes ago. However, the existence of this page is completely normal. The equivalent page on mastodon.social is mastodon.social/@dentangle@chaos.social, and it is how I can talk to you despite not having an account on chaos.social.
@Gargron @dentangle @jonny I'm aware of backfilling and profiles existing on fediverse instances. So are other folks.
My main issue is it being searchable on search engines. Plus mushing everything together without respecting the public/quiet public stuff.
Thank you Eugen for making them aware.
-
@dentangle @jonny @onepict As I said, that page should have a noindex tag on it (if you know what that is), and I consider it an oversight that it doesn't. I've let the NodeBB folks know about it a few minutes ago. However, the existence of this page is completely normal. The equivalent page on mastodon.social is mastodon.social/@dentangle@chaos.social, and it is how I can talk to you despite not having an account on chaos.social.
@Gargron @jonny @onepict Thanks. Yes, I understand. I do hope it is merely an "oversight" as you put it.
Given the number of times we've had to slap down attempts to make the fediverse searchable it's astonishing that a fediverse developer wouldn't take more care. Mistake or not, it's a huge breach of trust.
-
@Gargron @jonny @onepict Thanks. Yes, I understand. I do hope it is merely an "oversight" as you put it.
Given the number of times we've had to slap down attempts to make the fediverse searchable it's astonishing that a fediverse developer wouldn't take more care. Mistake or not, it's a huge breach of trust.
@dentangle @Gargron @jonny @onepict so at a protocol level "quiet public" doesn't really exist, all that happens in mastodon is that as:Public gets moved from `to`to `cc`, so they're effectively the same audience being addressed.
So NodeBB is actually right, at a protocol level, to treat public and "quiet public" as the same.
Though it sounds like steps will be taken to prevent indexing & display (when unauthenticated) of remote content outside of the context of a thread (you can't exactly mark sections of a page as noindex)
-
@dentangle @Gargron @jonny @onepict so at a protocol level "quiet public" doesn't really exist, all that happens in mastodon is that as:Public gets moved from `to`to `cc`, so they're effectively the same audience being addressed.
So NodeBB is actually right, at a protocol level, to treat public and "quiet public" as the same.
Though it sounds like steps will be taken to prevent indexing & display (when unauthenticated) of remote content outside of the context of a thread (you can't exactly mark sections of a page as noindex)
@thisismissem @Gargron @jonny @onepict
The problem, as Gargron identified, appears to be the lack of a "noindex" tag, which in Fediverse terms is like running an SMTP open relay - a misconfiguration rather than a fault in protocol - but which should not be the default in any software, and which will get you instablocked by the entire Internet.
