Is there any way to automate community discovery across lemmy instances?

I’m just getting my lemmy community going and I’m seeing super weird behavior when I’m trying to add communities or connect via federation.

  • dragontamer@lemmy.worldEnglish
    0·
    2 years ago

    I don’t think its a good idea to rely upon Lemmy search for community discovery. I agree that its where “users expect to go”, but there’s several downsides.

    1. Servers don’t know about other servers until someone on the server subscribes to them.

    2. After the subscription, the server eventually pulls in the data needed to hook up. I don’t know how long this takes, but its longer than “minutes”.

    For now, discussion-based discovery is how the community does things in practice. That is: !newcommunities@lemmy.world seems to be the pragmatic approach. Its a bit of a human-touch for now and not automated, but its probably our best bet for the near future.

    A google-like approach, where a web-crawler creates a search engine is what the community needs. The web-crawler could be a bot@some.instance, maybe its own instance for maximum “politeness”.

    • God@wallstreets.betOPEnglish
      1·
      2 years ago

      Yeah, this just makes it much more difficult for small, niche instances to grow. It focuses sign ups into monolith instances (since you don’t need to do discovery), which imo is pretty central to the issue reddit created in the first place.

      Personally, I don’t think lemmys like lemmy.ml, or lemmy.world, or beehub.org should really exist. The community duplication and centralization of content is going to represent a real barrier to effectively replacing reddit (as a viable option). I think it makes a lot more sense to have smaller lemmys focused on niche topics (for example, lofi, or sound engineering, or gardening, or in my case, degenerate financial advise). Each lemmy might be composed of a few communities which are all on theme, but can be federated more broadly into the entire ecosystem, so that cool niche things can be found.

      For this to function there has to be some kind of automated community discovery (like you mentioned with a crawler, which it sounds like what I’ll need to make for this to work.)

      Already were seeing the issue that the centralization of users and content have created. All the drama lemmy.world and beehub.org have caused. Lemmy.ml crashing and struggling to serve content. Its all a result of centralization, which imo, is antithetical to the principal of the fediverse.