Why is data congregation so hard on Mastodon?
-
This applies to any of the microblogging software. Akkoma, IceShrimp, etc. I go to any Lemmy instance, big or small, and the up/downvote data and replies are basically all the same. The same goes for Peertube, and most services that aren't Mastodon and the gang. Why is this? Is it because of older design? Unexpected issues cropping up with scale? It seems to be such a big struggle over there, but for everyone else, it's whatevs.
I would love to permanently reside on a smaller Mastodon instance or host my own, but I often find that many posts are unavailable and a lot of replies I want to reply to don't exist. It is an incredibly frustrating experience.
-
This applies to any of the microblogging software. Akkoma, IceShrimp, etc. I go to any Lemmy instance, big or small, and the up/downvote data and replies are basically all the same. The same goes for Peertube, and most services that aren't Mastodon and the gang. Why is this? Is it because of older design? Unexpected issues cropping up with scale? It seems to be such a big struggle over there, but for everyone else, it's whatevs.
I would love to permanently reside on a smaller Mastodon instance or host my own, but I often find that many posts are unavailable and a lot of replies I want to reply to don't exist. It is an incredibly frustrating experience.
Lemmy and lots of other software use a fediverse extension called 1b12 to keep everything in sync.
In a nutshell it means Lemmy communities can follow other communities, and they keep each other in sync. The same applies for other types of communities, like PieFed communities, Mbin magazines, NodeBB categories, etc.
Mastodon doesn't have a concept of community or categories, so they don't support this kind of synchronization.
-
Thank you, this makes sense. Is there any hope for Mastodon and other services to achieve a similar level of parity without eating up a ton of space? I feel like it is a big hurdle for Fedi, but I understand these things take time and a ton of work.
-
Thank you, this makes sense. Is there any hope for Mastodon and other services to achieve a similar level of parity without eating up a ton of space? I feel like it is a big hurdle for Fedi, but I understand these things take time and a ton of work.
Sure, check out my post about it here:
https://community.nodebb.org/topic/18844/backfilling-conversations-two-major-approaches
There are steps being taken in the right direction.
-
This applies to any of the microblogging software. Akkoma, IceShrimp, etc. I go to any Lemmy instance, big or small, and the up/downvote data and replies are basically all the same. The same goes for Peertube, and most services that aren't Mastodon and the gang. Why is this? Is it because of older design? Unexpected issues cropping up with scale? It seems to be such a big struggle over there, but for everyone else, it's whatevs.
I would love to permanently reside on a smaller Mastodon instance or host my own, but I often find that many posts are unavailable and a lot of replies I want to reply to don't exist. It is an incredibly frustrating experience.
Its all about pub sub relations. Being on a smaller instance in mastodon means u miss a whole bunch of activity and thus don't have the full picture.
Lemmy and others have a community which means a community receives an activity and then that community pushes that activity to all its subscribed instances. So in lemmy u need at least 1 user from ur instance to subscribe to a community for its activity to be sent to ur instance.
On mastodon all activity by a user is sent to all instances with at least one user who follows that user. So if nobody on ur instance follows a specific user then their content will never show up on ur instance. Its a difficult problem to solve as u can't just send all activity to all instances cos that will overload most instances. Their are relays that relay activities from its member instances to other instances but as u would expect the larger the relay the more content u must deal with on ur instance and u can get overloaded pretty quick.
The whole point of federation is to send activities only to instances that need them as not to force everyone to have the entire fediverse on their server and handle and activities from the entire fediverse (one of the reasons bluesky federation doesn't work for small instances).
-
Very interesting read!
-
This applies to any of the microblogging software. Akkoma, IceShrimp, etc. I go to any Lemmy instance, big or small, and the up/downvote data and replies are basically all the same. The same goes for Peertube, and most services that aren't Mastodon and the gang. Why is this? Is it because of older design? Unexpected issues cropping up with scale? It seems to be such a big struggle over there, but for everyone else, it's whatevs.
I would love to permanently reside on a smaller Mastodon instance or host my own, but I often find that many posts are unavailable and a lot of replies I want to reply to don't exist. It is an incredibly frustrating experience.
I started my own GoToSocial instance this week, and found the same. TBH, from my exported list of about 160 people I from followed on Mastodon, I only re-followed 6.
These were all longstanding relationship - a couple of which started on Twitter when it was OK.
-
This was very interesting! While I don't fully understand everything, based on what I can, I'm partial to the second one. If the the instance disappears, I doubt the hoster wants that stuff up anyway. It makes it easier on everyone, and the replies seem (?) to stay up as well. A win-win. Either way it goes, I can only be thankful to the engineers working so hard to make this a reality.
-
This was very interesting! While I don't fully understand everything, based on what I can, I'm partial to the second one. If the the instance disappears, I doubt the hoster wants that stuff up anyway. It makes it easier on everyone, and the replies seem (?) to stay up as well. A win-win. Either way it goes, I can only be thankful to the engineers working so hard to make this a reality.
Thanks! It's something that I personally feel is more performant and future proof for other important things like private discussions (which Mastodon also doesn't support natively yet β mention spamming doesn't count.)
