Lemmy AMA March 2025
-
20 GB RAM for a single user instance sounds like a lot. Did you use pgtune? It may also help to run a reindex or full vacuum.
Yeah I used pgtune as a base and found more memory needed to be assigned to certain spots especially to keep federation with bigger instances, otherwise timeouts would occur resulting in my instance being constantly behind.
That said I read postgres 17 is much more memory efficient, though I have yet to move my lemmy database to it yet since its the largest haha.
-
They did managed hosting before but everyone freaked out that the communist devs were controlling the narrative or some shit. It was a mess, I think they stopped doing it now that there are more instances.
Down votes should remain. It expresses dissatisfaction with peoples cringe bad takes
We offered free instance hosting for some time, in order to encourage the creation of new instances. That was before the Reddit migration when lemmy.ml was still by far the largest instance. It was a success because numerous instances like slrpnk.net and aussie.zone were started that way. But after the migration there was no need for it anymore as plenty of instances were created without our help.
-
- From a code architecture perspective, how close is Lemmy/ActivityPub to reaching its maximum capacity for posts/comments per second? Are there any ways to 10x the load ActivityPub can handle?
- With Nicole in everyone's DMs, what does the future of spam filtering look like on Lemmy?
- There is no specific maximum capacity, in theory it can scale indefinitely with horizontal scaling.
- 0.19.10 already includes a fix to remove private messages when a user gets banned which should help a lot. There is an issue about disabling private messages by default, but Im not sure if that will be necessary. Also 1.0 will include a plugin system, so other devs and instance admins can write their own checks. That way spam waves can be fought in a more flexible way, without having to get a change merged into Lemmy and then waiting for a new release.
-
What have been the biggest challenges with the project over the years, both in terms of technical and non technical aspects. I'd be interesting to hear a bit of retrospective on how has the stack's been working out, and what surprises you might've run into in terms of scaling and federation. What recommendations you'd make based on that and what you would've done differently knowing what you know now.
The stack is great, I wouldnt want to change anything. Postgres is very mature and performant, with a high focus on correctness. It can sometimes be difficult to optimize queries, but there are wizards like @dullbananas@lemmy.ca who know how to do that. Anyway there is no better alternative that I know of. Rust is also great, just like Postgres it is very performant and has a focus on correctness. Unlike most programming languages it is almost impossible to get any runtime crashes, which is very valuable for a webservice.
The high performance means that less hardware is required to host a given number of users, compared to something like NodeJS or PHP. For example when kbin.social was popular, I remember it had to run on multiple beefy servers. Meanwhile lemmy.ml is still running on a single dedicated server, with much more active users. Or Mastodon having to handle incoming federation activities in background tasks which makes the code more complicated, while Lemmy can process them directly in the HTTP handler.
Nevertheless, scaling for more users always has its surprises. I remember very early in development, Lemmy wasnt able to handle more than a dozen requests per second. Turns out we only used a single database connection instead of a connection pool, so each db query was running after that last one was finished, which of course is very slow. It seems obvious in retrospect, but you never notice this problem until there are a dozen or so users active at the same time.
With the Reddit migration two years ago a lot of performance problems came up, as active users on Lemmy suddenly grew around 70 times. You can see some of that in the 0.18.x release announcements. One part of the solution was to add missing database indexes. Another was to remove websocket support, which was keeping a connection open for each user. That works fine with 100 users, but completely breaks down with 1000 or more.
After all there is nothing I would do different really. It would have been good to know about these scaling problems earlier, but thats impossible. In fact for my project Ibis (federated wiki) Im using the exact same architecture as Lemmy.
-
Some Lemmy clients offer the option to auto-hide posts and comments which contain certain keywords of the choice of the user. Are there any plans to implement this feature into the stock Lemmy experience?
I know it is possible to do some hacky stuff with UblockOrigin to do the same, but that is not something most know about and are willing to do.
This is currently work in progress.
-
From my perspective we need better Mod and Admin tools. Forum software has a lot of them but Lemmy is lacking in this department.
The key important one is being able to move posts to different communities. You'll often get reports of posts not being appropriate for a community but there is no way to actually move it.
There was some discussion of this not long ago: https://feddit.uk/post/24412286
@nutomic@lemmy.ml linked this GitHub issue: https://github.com/LemmyNet/lemmy/issues/2345
-
Edit: I suppose I shouldn't be answering this. Kinda forgot the thread I'm in. I guess I asked something as well.
If your instance was federated with it when it existed then your instance automatically has its own backup of it is as far as I understand things. I would like clarity on this however. My instance is a few days older than this account. Therefore the smaller instances that have already died are already duplicated locally here at sh.itjust.works. I can still view vlemmy, waveform.social, lemmy.film, (etc.) communities/posts as essentially an archive.
What I'd like to know is if I linked a sh.itjust.works link to one of those threads could a user of a more recent instance load the content?
I'm not sure what point it would ultimately serve as with the host instance being offline nothing could federate out between us anyway.
This, content is already mirrored to federated instances and stored forever (though media may not be included).
What I’d like to know is if I linked a sh.itjust.works link to one of those threads could a user of a more recent instance load the content?
Lemmy only loads content from the original instance where it was created, otherwise it would be possible to impersonate users. So it is not possible to load that.
-
-
Will there be any way to block users from certain instances to hide their comments?
-
What are the plans the improve discoverability?
I'm quite discontent with how few options there is to explore Lemmy. And it doesn't helps that the top posts are always related to politics.
- Will there be any type of word filtering?
We should have community unifying.
- I know people have already said it many times, but the joining experience could be simpler and less confusing.
Will there be any way to block users from certain instances to hide their comments?
I dont think theres an issue for this yet, feel free to open one. It could be a checkbox for "Blocked Instances" setting, eg "Also block users".
What are the plans the improve discoverability?
There is an issue for easier discovery of federated communities which is part of our roadmap. Piefed recently implemented a similar feature which we will take inspiration from. It also helps if you block communities that you dont want to see. Are there any other ideas you are thinking of?
Will there be any type of word filtering?
This is work in progress.
-
-
There was some discussion of this not long ago: https://feddit.uk/post/24412286
@nutomic@lemmy.ml linked this GitHub issue: https://github.com/LemmyNet/lemmy/issues/2345
It shouldn't be too difficult. A move is essentially a cross-post but it keeps the OP as the poster (rather than the cross-poster). You'd then want to lock the original post, and either hide it or add a message directing people to the new post. That's all current forum software does.
-
Yeah I used pgtune as a base and found more memory needed to be assigned to certain spots especially to keep federation with bigger instances, otherwise timeouts would occur resulting in my instance being constantly behind.
That said I read postgres 17 is much more memory efficient, though I have yet to move my lemmy database to it yet since its the largest haha.
Maybe your disk is too slow, or latency between Lemmy and Postgres is too high?
-
Maybe your disk is too slow, or latency between Lemmy and Postgres is too high?
It is a k8s cluster and using ceph for all of my storage so the latency from that I bet is the largest reason and upping the memory offsets the disk writes. i also have another postgres DB syncing as a fallback for high availability. Fortunately after tuning the database and giving it enough RAM my instance has been running pretty stable for over a year without any changes.
I am also using less powerful computers for the entire infrastructure (not server grade) which brings to the point of having horizontal scaling on database I imagine will be a growing need with growing instances, communities, and users since it can be cheaper to run multiple smaller spec servers rather than a single with the added benefit of high availability.
-
It shouldn't be too difficult. A move is essentially a cross-post but it keeps the OP as the poster (rather than the cross-poster). You'd then want to lock the original post, and either hide it or add a message directing people to the new post. That's all current forum software does.
The main question is how this can work in terms of federation. When creating a new post it directly references the community url. If the user and community are on different instances then the community instance cannot rewrite the post to reference a different community. So it would have to tell the post creator to (automatically) resubmit the post to the new community. Same for all comments, they would have to be recreated by the respective author's instance in the new post. Seems quite complex to implement.
-
What’s something you wish it had? What can our community do to ensure that we keep pulling users away from US tech companies, and into the fediverse?
One of the biggest issue at this point is probably the registration experience. There are quite a few occurrences on !fedibridge@lemmy.dbzer0.com of users not sure whether their email has been validated or not, and at the moment they really need to look out for the toastify notification on their first try, later attempts won't show it.
Most recent example: https://lemmy.ml/post/27607055?scrollToComments=true
If there could be a way to inform a user saying "your email address has been validated, please wait for an administrator to activate your account, you can reach out to them at xxx", that would be great.
Youre right, I also noticed some other problems while testing registrations:
-
Is there a way to move myself as an user from one server to another?
You can export your settings, community follows etc and import them in another instance. Moving your existing posts and comments doesnt work well with federation.
-
Some companies use Reddit as their main forum or an established way to communicate with customers. Are there any companies that have explored Lemmy and have their community yet?
Not that I know of, only some open source projects as mentioned by others.
-
In the last weeks Lemmy has seen a lot of growth, with thousands of new users. To welcome them we are holding this AMA to answer questions from the community. You can ask about the beginnings of Lemmy, how we see the future of Lemmy, our long-term goals, what makes Lemmy different from Reddit, about internet and social media in general, as well as personal questions.
We'd also like to hear your overall feedback on Lemmy: What are its greatest strengths and weaknesses? How would you improve it? What's something you wish it had? What can our community do to ensure that we keep pulling users away from US tech companies, and into the fediverse?
Lemmy and Reddit may look similar at first glance, but there is a major difference. While Reddit is a corporation with thousands of employees and billionaire investors, Lemmy is nothing but an open source project run by volunteers. It was started in 2019 by @dessalines and @nutomic, turning into a fulltime job since 2020. For our income we are dependent on your donations, so please contribute if you can. We'd like to be able to add more full-time contributors to our co-op.
We will start answering questions from tomorrow (Wednesday). Besides @dessalines and @nutomic, other Lemmy contributors may also chime in to answer questions:
Here are our previous AMAs for those interested.
How is it some can mod 15+ comms, like this awful character PugJesus , ban anyone for no reason and then comment stuff like this without consequence:
Be less of a dick.
Be less of a moron. -
How are you?
A bit tired because my whole family is half sick. Luckily the kids are still okay to go to school.
Otherwise Im excited for this AMA, because I rarely have such direct conversations with users about Lemmy. The discussions on Github are usually quite technical.
-
I'm just a user and don't follow the project super closely so please forgive me if this has already been addressed somewhere.
Is there any sort of JavaScript-less interface that would work properly on the Tor network with strict settings enabled? And could you set up instances only with a .onion domain? That way you don't have to pay for a domain and you're not at risk of having your domain yanked by ICANN, etc.
If I remember correctly, there was a mastodon instance that was using like a Pakistan domain or something like that and they yanked it.
Also, Federation between Onion and Standard Domains that way tor users would not be isolated.
My main reason for asking is that in my worldview, governments want to break encryption and break freedom of speech, if at all possible, and so the dark web is going to be more and more necessary as time goes on.
Edit: The more standard traffic we add onto the Tor network, the less able it is to be blocked or surveilled as to why somebody is using the Tor network. As an example, I send all of my signal traffic through Tor and download apps from fdroid through Tor and chat on SimpleX through Tor and quite a number of other things. Not because I need the Tor network, but just simply to be yet another person using it for standard activity
There is documentation for running a Lemmy instance over Tor, and one of the many frontends probably works without JS. If not someone could implement it with the API. Anyway there doesnt seem to be much interest in practice, because the clearnet works good enough for now.
-
To chime in on the user creation thing:
I think it's a natural part of decentralization that it's harder for a single instance to get big enough to be the "go-to" for general users.
Having said that, I also think this will naturally happen over time. As long as the mechanical aspects of sign up are simple, it's just a matter of users of a given instance to promote their instance.
World events also always play a role in encouraging a move to freer waters. Look at what happened with Mastodon and Bluesky (though Bluesky imo is just a big snooze button on a blaring alarm)
Exactly, the more time goes by the better Lemmy will get. For sites like Reddit or now Digg its much easier to do marketing and get a quick user growth, but when they have problems then users will move to Lemmy.
-
There is documentation for running a Lemmy instance over Tor, and one of the many frontends probably works without JS. If not someone could implement it with the API. Anyway there doesnt seem to be much interest in practice, because the clearnet works good enough for now.
Okay, that shows how to make the interfaces available over the network. But is there planned functionality for making Federation work? Even if you don't federate to clearnet domains, you could have several onion instances that could federate with each other and have a network of onion instances.
