When querying for “legal” and “law” on lemmyverse.net, it’s starkly clear that many communities are missing from the database. I get far more results when querying on specific instances. So what’s the problem? Is it no longer crawling?
When querying for “legal” and “law” on lemmyverse.net, it’s starkly clear that many communities are missing from the database. I get far more results when querying on specific instances. So what’s the problem? Is it no longer crawling?
I understand the confusion.
Maybe I should’ve been clearer that lemmyverse performance has been unreliable, not just the crawler. Entire instances, communities, and mixes of the two have disappeared and reappeared between scans. It also hasn’t been reliably picking up changes to instances it already knows about.
TLDR lemmyverse has something wrong with it.