It is fairly relevant to lemmy as is. Quite a few instances have ram constraints and are hitting swap. Consider how much worse it would be in python.

Currently most of the issues are architectural and can be fixed with tweaking how certain things are done (i.e., image hosting on an object store instead of locally).

On the other hand, Rust is fairly resilient. The issues Lemmy is experiencing wouldn’t be fixed in Python vs Java, it’s more of an architectural constraint. Those issues, experienced devs can fix mostly regardless of language.

Sorry if I was curt! No reason to be sorry for throwing out a decent idea

Caddy is not going to fix anything, on the contrary, it consumes more ram. Generally the instances have been slowing down when swap gets hit by the db, so lowering ram usage and optimizing that should be the first priority.

The issues I’ve seen more are around images. Hosting the images on an object store (cloudflare r2, s3) and linking there would reduce a lot of federation bandwidth, as that’s probably cause higher ram/swap usage too.

pict-rs supports storing in object stores, but when getting/serving images, it still serves through the instance as the bottleneck IIRC. That would do quite a bit to free up some resources and lower overall IO needed by the server.

I will be working on this when I get cycles. Barring the issues already above, there are a lot of areas for optimizations, for instance how images are handled (i.e., they can be handled through object storage like Cloudflare R2 to decrease bandwidth/ram costs). Some is more dev-ops on how common instances are setup, others are code changes to make things more efficient.

Perhaps we should start a community or communication group for this?

+1, lemmy/kbin/mastodon are communities users can shape and contribute to (literally to the code as well) far more than reddit. That alone with along with the recent influx of users, makes it a far more interesting place than reddit.

Context size is huge, as well as the ability to context switch effectively. It can mean the difference between solving something in a day or weeks.

I like problem decomposition a lot as a discrete step. There’s a huge tendency to go, I have problem A, let’s just solve with it B. Many times the nuance of why A occurred, whether it’s a symptom of something, and what are the different subproblems that comprise A are skipped.

This often causes solutions which don’t actually solve the problem, or just mask it. That extra effort up front, leads to the proper solution, and as you said, very tactical fixes instead of huge unnecessary solutions.

Definitely agree there! Communication is super underrated, especially with how difficult it can be to align people and teams across organizations.

Jargon is great for consolidating complexity into just a few words, reducing the things you have to think about. It can be equally valuable though to poke into implicit assumptions that are commonly made.

It’s definitely a balance, and being inclusive in conversations is super important as you mentioned. It allows newer folks to get up to speed much faster in comparison, and allows more engagement across the people within the discussions.

Oh absolutely! Looking to get it running locally so I can start building something out and trialing something. Trial and error iteration would work well here I think

What API are you using? Where is the data stored? Would be easier to answer with more data on how this is setup.

The tricky part is defining an algo for hot, it’s a bit different than reddit since some instances may have many people, and local may have fewer, so there should be a balance such that posts from larger instances don’t overwhelm local instances perhaps.

Indirect prompt injections will make this worse. Plugins lead to scraping insecure websites (i.e., search for docs for a particular topic). This can result in malicious context being embedded and suggested during a PR or code output.

That along with the above, faking commonly recommended inputs, it becomes very difficult to just trust and use LLM output. One argument is that experienced devs can catch this, but security is often about the weakest link, one junior dev’s mistake with this could lead to a hole.

There are guard rails to put in place for some of these things (i.e., audit new libraries, only scrape from ‘reliable’ websites), but I suspect most enterprises/startups implementing this stuff don’t have such guard rails in place.

Related

• https://medium.com/palo-alto-networks-developer-blog/mind-tricks-the-perils-of-prompt-injection-attacks-against-llms-4b148fcd7519

• https://rez0.blog/hacking/2023/05/19/prompt-injection-poc.html#:~:text=Indirect Prompt Injection%3A A technique,for the next LLM execution.

I think there are people working on adding support for pgbouncer and splitting out pg from the core server to avoid having a 1 box only setup.

Soft fork works like this:

1. Create fork
2. Update with whatever features are needed
3. Try to upstream to main via upstream pull request
4. Rebase off of original repo
5. Repeat from 2

In practice, there shouldn’t be much of a diff, so it will be easy to keep it up to date (as long as original repo accepts upstream PRs from the soft fork). There are some minor features which need to be added, and it helps the original repo anyways.

That’s why a soft fork is a good idea as well, there will be multiple groups evolving lemmy and have expertise to solve issues or work on features as they arise.

If one group goes crazy, the other can still continue supporting.

Doesn’t support HA or horizontal scaling yet from what I read. Unsure if kbin does. Probably would have to add support for horizontal scaling to have that auto scaling do anything.