I gather that it had a use unrelated to Lemmy but for Lemmy posts I make a case here that it is more misleading than helpful. For a moment, please ignore the underlying reasons why things are the way they are and focus on how the issue presents to the end-users.
(1) By pulling in solely the post, but not any of the comments, it at best provides only partial information - which if all you wanted to read was the post, then why bother pulling it here at all? (as opposed to retrieving from its original location - I mean, to do it you already need the full URL…) While if instead you wanted all of the comments… - e.g. to be able to reply to - then too bad, b/c it won’t do that?
(2) It also does not pull in any of the old vote counts. So if hypothetically a post had 1000 upvotes, and then after pulling it here it received adjustments +2 from upvotes and -4 from downvotes, then its total would then be 998, right? Except PieFed would instead display “-2”, a qualitatively different score for a highly popular post that is a terrible misrepresentation of the actual facts about it.
(3) It conveys a distorted view of things to the end-users. e.g. see !yepowertrippinbastards@lemmy.dbzer0.com where there are 6 posts from the last 2 months, right? Right?! No, there is actually only a single post there in its entirety, then a few more that I and what I assume was Blaze pulled in - note how those other 5 have zero comments, and total scores near zero, due to the aforementioned issues. Really the “earliest” post that PieFed.social reliably has from that community is from 4 days ago, and then beyond that is a scattered, partial mess. There are actually MANY more posts from the last two months, which are not represented here. Ergo, the initial impression that a quick glance at this community offers turns out to be false, due to these federation issues.
(4) showing only partial information is often called a “false positive” or type I style of error, whereas showing nothing at all for those posts that are not fully here avoids that pitfall. If certain content is not here then… well it is not here, and that’s that, but for only some of it to be here leads to much confusion, imho.
Almost entirely distinct from this issue, the ability to find an existing post given its URL should be added to the search menu, b/c that is where people will go to find it. But ofc all the more so if the retrieval button is removed or made less prominent, so that that find ability is not lost along with that.
I understand that there are hard limitations of the federated model itself. So if e.g. older comments and votes cannot retroactively be pulled in - or possibly even if so - then maybe this function should just be abolished? Or perhaps a couple more layers of “are you sure you want to do this?” added, or better yet moving it from its prominent place showing up to everyone on almost every page to a more subdued location where only those who know what it is and what will happen if it is used are likely to access it? I now feel that I actively made the situation in !yepowertrippinbastards@lemmy.dbzer0.com worse by pulling in those posts, and wished now that I hadn’t done so, as it could lead people astray into thinking “this is all the posts that the community has to offer from this time-period” (NO, it actually has MANY MANY more than that, on the original server!?!!). Now that I know this I can refrain from using it, but it would be nice to help others who climb this ladder after me as well:-). So I am sharing my thoughts with you in case that helps.
PieFed is freaking awesome and you all who work on it are magnificently extraordinary to share your knowledge with the world:-).
Part 2 of 2.
That said: I thought that we have a problem that Lemmy does not: we allow users to pull in older posts. But your linked example seems to disprove that - do you know how they got those posts then, even though they are “old”? Or perhaps I am misunderstanding something: we CAN get older posts, but we usually choose NOT to, UNLESS someone specifically requests the pulling in of such a post. Damn it does get complicated:-). In general though, I hope you see what I meant to convey: if we can only get new posts and cannot get old ones accurately, then perhaps it is better not to try to get those old ones (except as ghosts/shortcut links)?
As for existing backfilled posts - yeah, it is unfortunate, but it is what it is and I don’t think “backwards compatability” to existing entries is our primary concern right now? If it were, they could perhaps be deleted - if we could even figure out which ones they are? - or left alone. A year from now they will confuse the heck out of people… but that can be an adventure for them to puzzle over, so long as things work properly from here forward - do you agree? :-D
Anyway it’s just something fun to think about:-).
Hey, thank you for this engaging set of posts - and the entire post and thread, really. It really opened my eyes up and made me think hard - exploring a new mental space, in Star Trek speak ( see https://memory-alpha.fandom.com/wiki/All_Good_Things…_(episode)#Act_Ten for the reference - it happens near the end of the episode)
I think I prefer 1 or the ? version. Going with 2 means I can’t interact with a post that fails to federate (or is older than my instance) at all. With 1 or ? I can. Being able to backfill comments would be icing on the cake.
From the linked post of https://discuss.tchncs.de/post/1907501 it seems the standard bearer is just what the home instance of the community has for that post. So comments and votes made to a post after defederation do not count. This makes sense - if you can’t trust the other instance, probably you can’t trust their admins either - and it’s easy for an admin with database access to add fake comments and up/downvotes, etc. So we just pick the obvious source of truth and roll with that.
One small issue here - from screenscraping we can get the vote total count but maybe not the individual votes. We wouldn’t know who voted which way. That’s not a big deal - we can just make up a “random0123456@example.com” ap id to own the vote in the backfilled cases, but I think it makes the case for ? rather than 1.
(As an aside I don’t think Lemmy does backfilling, only pyfedi does it - as the top comment says, “just… it does not backfill previous content and votes.” But the post itself I think must get picked up when a new comment or up/downvote gets seen.)
Damn I loved that Star Trek episode - it was so great! It’s so easy to just keep doing the same old thing - e.g. to explore the stars, but to join a card game - THAT would be a NEW adventure, surely.
But as for the possible, one idea would be to make a hard separation: let’s say a PieFed user joins a new community XYZ. As of that very moment, all newly made posts along with all votes come in, and everything that can make it there through any federation networking issues are reflected in that posts, BUT all old posts do not show up - even if new votes or comments are sent, those are simply discarded - and perhaps a message is displayed “there are no more posts available from this community”. So everything NEW is fully there (to the extent possible/feasible), while everything OLD is fully missing (it would have been only partial/misleading information anyway), and this way there is nothing in-between.
At least, it’s a thought:-). Surely it would not please everyone, but it might be a least worst option?
Err - so basically do what Lemmy does?
Like I said, that wouldn’t work for me. But then again, I’m an admin of my single user instance. And I can see the needs of “real” instances - ones trying to build a community of many users - being different from mine.
So maybe allow it as an admin toggle, default to Lemmy behaviour. That would seemingly cover everyone - and the admins on piefed.social can manually pull in an older post/comment if there’s a compelling reason to make an exception (and add the cautions about possibly inaccurate vote counts and such).
No (I think?) - I thought your link showed that Lemmy was showing inaccurate information as well. So what I was proposing was perhaps better than what Lemmy does, in the sense that Lemmy would pull in the additional posts upon a new voting occurrence, but PieFed could block that and either show only posts for which fully accurate information is available, and then discard those where the extra (voting) information is not available, or else it could go a step further and go ahead and show those posts that only have partial information but make sure to label it specially to indicate that. Lemmy does neither of those.
Second issue: right - an admin should be able to, and maybe there’s no real reason to prevent regular users either, but it shouldn’t be so common, so it would be nice to move the link to do that somewhere else so that new users (like me!) don’t get confused about what it means and is supposed to do and pull things in on a whim without knowing the factors involved.
On the 1st point - well yes, Lemmy isn’t completely accurate either. But that’s inherent to federation. The suggestion is a good one, but consider this:
Showing that only partial info is available is possible, just compare to the home instance’s page via screenscraping and throw up the partial info banner if the counts look different.
Beyond that, discarding the extra comments and partial voting info that could be retrieved via screenscraping is basically what we do now - it’s literally just doing nothing with the extra info. By contrast, doing stuff like figuring out the comment URL or ap_id and stuffing it into a database, or creating dummy actors to store partial voting records would actually require extra effort.
So “1” is impossible, and “?” requires a lot of smart effort, while “2” is the easiest. Usually the impossible things are the ones most worthwhile to build.
On the 2nd point, agreed. Perhaps there is a better place to put that link - could just live somewhere on the Community’s page rather than on each on and every post.