I gather that it had a use unrelated to Lemmy but for Lemmy posts I make a case here that it is more misleading than helpful. For a moment, please ignore the underlying reasons why things are the way they are and focus on how the issue presents to the end-users.
(1) By pulling in solely the post, but not any of the comments, it at best provides only partial information - which if all you wanted to read was the post, then why bother pulling it here at all? (as opposed to retrieving from its original location - I mean, to do it you already need the full URL…) While if instead you wanted all of the comments… - e.g. to be able to reply to - then too bad, b/c it won’t do that?
(2) It also does not pull in any of the old vote counts. So if hypothetically a post had 1000 upvotes, and then after pulling it here it received adjustments +2 from upvotes and -4 from downvotes, then its total would then be 998, right? Except PieFed would instead display “-2”, a qualitatively different score for a highly popular post that is a terrible misrepresentation of the actual facts about it.
(3) It conveys a distorted view of things to the end-users. e.g. see !yepowertrippinbastards@lemmy.dbzer0.com where there are 6 posts from the last 2 months, right? Right?! No, there is actually only a single post there in its entirety, then a few more that I and what I assume was Blaze pulled in - note how those other 5 have zero comments, and total scores near zero, due to the aforementioned issues. Really the “earliest” post that PieFed.social reliably has from that community is from 4 days ago, and then beyond that is a scattered, partial mess. There are actually MANY more posts from the last two months, which are not represented here. Ergo, the initial impression that a quick glance at this community offers turns out to be false, due to these federation issues.
(4) showing only partial information is often called a “false positive” or type I style of error, whereas showing nothing at all for those posts that are not fully here avoids that pitfall. If certain content is not here then… well it is not here, and that’s that, but for only some of it to be here leads to much confusion, imho.
Almost entirely distinct from this issue, the ability to find an existing post given its URL should be added to the search menu, b/c that is where people will go to find it. But ofc all the more so if the retrieval button is removed or made less prominent, so that that find ability is not lost along with that.
I understand that there are hard limitations of the federated model itself. So if e.g. older comments and votes cannot retroactively be pulled in - or possibly even if so - then maybe this function should just be abolished? Or perhaps a couple more layers of “are you sure you want to do this?” added, or better yet moving it from its prominent place showing up to everyone on almost every page to a more subdued location where only those who know what it is and what will happen if it is used are likely to access it? I now feel that I actively made the situation in !yepowertrippinbastards@lemmy.dbzer0.com worse by pulling in those posts, and wished now that I hadn’t done so, as it could lead people astray into thinking “this is all the posts that the community has to offer from this time-period” (NO, it actually has MANY MANY more than that, on the original server!?!!). Now that I know this I can refrain from using it, but it would be nice to help others who climb this ladder after me as well:-). So I am sharing my thoughts with you in case that helps.
PieFed is freaking awesome and you all who work on it are magnificently extraordinary to share your knowledge with the world:-).
Oh I never replied to this - I must have gotten distracted, sorry for ignoring you (unintentionally I assure you:-).
I agree that it is nice to “view” a post in its entirety from another server, but what is the benefit to pulling the post into here? Lemmy solves the former issue by adding the rainbow-colored federation icon to everything - posts, comments, users, all of it will go to the original instance. PieFed makes it quite difficult imho to do all of that. There is an EXTREMELY buried “View community on original server”, which actually looks like it is somehow tied to the list of moderators, and only found on a community page. There is no way afaict to go directly to an original actual post though, or comment. There is also a way to go to a users’ home instance, but not to the actual user account itself, again afaict.
I would love to see a retrieval function that puts the post into a temporary location and allows someone to “view” it for awhile, before discarding it later (a week? a month?). But there isn’t much benefit to this, since even without an account you can view any OP from its home instance already - so it’s a lot of work for very little gain.
But to have it “here” - I think the benefit would be that you could respond to the comments within it? Yes, that may be worth the disadvantages that I mentioned. Perhaps then it could be indicated with an icon, or a horizontal separator between it and the other more recent ones, to distinguish it as something pulled in specially rather than having arrived the usual way - e.g. if you look through a community and see posts from, let’s use a hypothetical scenario, “the last 2 months”, and then all of a sudden there is a jump and a single post from “2 years ago” appears, it may help to show that discontinuity? But anyway yes, being able to respond to comments in it does add functionality, so that I like.:-)
(Edit: except if the vote counts are off - that could be quite damaging? e.g. a post with -2 total score is VERY different than one with +998, so it tells an entirely different story in terms of community receptivity to that idea. I would rather see no score at all, like literally perhaps an “n/a”, rather than see incorrect values there. Lack of information is fine - so long as it is labelled - it is misinformation that is what causes the damage in terms of how a message is received, differently than what was intended originally to have been conveyed.)
No worries about the delay. Once a reply to me from someone didn’t federate and I didn’t see it by chance until a month later - see https://lemmy.world/comment/12146899
I think you are asking for something different actually. This feature that I like and use is different from what you’re asking - in some ways they’re the exact opposite. But at the same time they aren’t mutually exclusive - in fact I’d say they are complimentary.
But if I’m understanding correctly, it’s easy enough to add into piefed the "See original post at " thing. For a rough job, I just put this line below
[HTML_REMOVED]See on original instance at [HTML_REMOVED]{{ post.ap_id }}[HTML_REMOVED][HTML_REMOVED]
above this bit, https://codeberg.org/rimu/pyfedi/src/branch/main/app/templates/post/_post_full.html#L2
And then every post on my instance will take me to the original author’s instance. (There’s probably a bit of tweaking to do here - for example I put it above the “Home > Communites…” line as a quick hack which probably isn’t the most user friendly UX.)
I put a small diff file at the end of this post as well.
But - back to the original feature - I think you’ve got exactly what the benefit is. Here’s a real life example of how it was useful:
For a while, I had issues with federation that was causing posts from lemmy.world not to federate to my instance for three whole days. But with this option, I could manually load in posts into my instance and add comments (and for some reason the replies to those comments tended to come in right away). Without this option I wouldn’t have been able to comment at all unless I waited, or used an account on a difference instance.
I’m fine with storing extra metadata so the post original (typical federation vs being manually pulled in - and perhaps a third for posts that get backfilled when a new community is first added?). In fact your suggestions on displaying this to the end user, like a special icon, are good ones.
But keep in mind that the vote discrepancy already exists on backfilled posts, and note that it could also happen with something that arrives through normal federation.
(E.g. Beehaw defederated from lemmy.world at some point but AFAIK piefed.social can still talk to Beehaw. Then up/down votes from users on Beehaw won’t make it into the vote count on the version of the post on lemmy.world but would make it to piefed.social - which might not be that big of a deal if it’s just a couple of users, but if the difference was massive then we’d have exactly the problem you state.)
We wouldn’t be the first to notice this either. See for example https://discuss.tchncs.de/post/1907501
diff --git a/app/templates/post/_post_full.html b/app/templates/post/_post_full.h
tml
index 3467929…df06190 100644
-– a/app/templates/post/_post_full.html
+++ b/app/templates/post/_post_full.html
@@ -1,4 +1,5 @@
[HTML_REMOVED]
+ [HTML_REMOVED]See original on [HTML_REMOVED]{{ post.ap_id }}[HTML_REMOVED][HTML_REMOVED]
{% if post.type == POST_TYPE_IMAGE -%}
[HTML_REMOVED]
[HTML_REMOVED]
Part 1 of 2.
In the past I have used the rainbow-colored Fediverse icon for things like a quick short-cut to the original instance, to check things like the community or instance rules, but mostly ever since Lemmy started being able to read the static.wikia types of images, I haven’t needed it anymore, now that all instances display more or less identical content for a given post.
With the exception of that known Lemmy.World federation issue. What happened there, from what I hear, is that the Fediverse was never meant to have a singular server holding 80% of the entire userbase, and to have every community be joined by someone on every instance (which Blaze has been doing with his army of alts, to aid newcomers who couldn’t figure out how to join a community that nobody else on the instance had already joined yet). Read more here: https://aussie.zone/post/13429731 (github issue linked from there). TLDR: the issue is already solved (probably) but is in the 0.19.6 version, which Lemmy.ml is testing but until Lemmy.World upgrades to it this issue will continue. I’ve seen it happen on piefed.social, but also startrek.website, discuss.online, and other servers as well.
But in general, I was not asking to be able to view every single post from its original server (though again, that would be somewhat useful, occasionally), nor even being able to pull in a post when it is not there, but to make the pulling in of a post be either full or else none. Here is a simple mathematical equation that may perhaps help: 1 + 1 = 2 is true, and 1 + 1 = ? could also be true, but 1 + 1 = 1 is not. The first one is fully and precisely true, the second is vague and unsatisfying that someone doesn’t know the answer… but at least it is still technically correct. Whereas the third answer is just flat false.
It sounds like from what you and others are saying that it is literally impossible to be able to get the answer of “2” via the current methods of Federation. Okay then, well that seems to inform our answer: if we absolutely cannot get the proper answer of “2”, then we should put “?”, but in no case should we (imho at least) show “1”, just b/c after we pull in a post it receives +1 upvote. “x+1=1” is invalid for nearly all values of x, except the one that happens to be true, and yet ALL of the others are false, hence we should not shorten x+1 to merely say “1” as if 1 + 1 = 1 were a correct statement.
Alternatively, which depending on interrelation with other reasons might be the better way even, perhaps we should simply disallow the pulling in of posts altogether - if they cannot be done “properly” then perhaps to avoid misleading aka false statements, it should not be done at all?
Still further alternatives could be to (1) pull in “ghost” or “shortcut” versions of the post, which are displayed VERY noticably different than “real” ones, and which when clicked go to the original server, and which also have something like “?” or “n/a” rather than a vote count; or (2) similar to 1 but which when clicked show all of the comments, those having been pulled in via some other means (API?) but there too the vote count(s?) should not be displayed, unless they are known to be accurate.
Other matters also interrelate with the above though. e.g. I notice that piefed.social has defederated from hexbear.net (https://piefed.social/instance/hexbear.net does not resolve and https://piefed.social/instances?search=hex&submit=Search looks empty) - so if any votes can from that instance, those should not be counted. Web scraping would include them though, and others e.g. those from people who piefed.social (or any PieFed instance, just picking on that one:-) has banned. Perhaps this is a minor point, but it does touch on how those vote counts are interrelated to other matters, which if brought in via “alternative” means will make it harder to figure out what to display for them. And… oops, I see that you already got to the reverse of that point, where Beehaw would add vote counts, beyond what e.g. web scraping would be able to pick up. Well, nothing is fully perfect, but I did want to suggest that we avoid any KNOWN sources of misinformation. Some corner cases like that… can be saved for perhaps another day when further developing this ActivityPub implementation makes more sense, compared to other more urgent things that need to be worked on in the meantime. i.e., especially if something is also a problem for Lemmy, then it seems understandable to me that if it is likewise a problem for PieFed, then there is little “expectation” that we would do better than them, in every single possible way?
Damn, I’ve hit a space limit.
Part 2 of 2.
That said: I thought that we have a problem that Lemmy does not: we allow users to pull in older posts. But your linked example seems to disprove that - do you know how they got those posts then, even though they are “old”? Or perhaps I am misunderstanding something: we CAN get older posts, but we usually choose NOT to, UNLESS someone specifically requests the pulling in of such a post. Damn it does get complicated:-). In general though, I hope you see what I meant to convey: if we can only get new posts and cannot get old ones accurately, then perhaps it is better not to try to get those old ones (except as ghosts/shortcut links)?
As for existing backfilled posts - yeah, it is unfortunate, but it is what it is and I don’t think “backwards compatability” to existing entries is our primary concern right now? If it were, they could perhaps be deleted - if we could even figure out which ones they are? - or left alone. A year from now they will confuse the heck out of people… but that can be an adventure for them to puzzle over, so long as things work properly from here forward - do you agree? :-D
Anyway it’s just something fun to think about:-).
Hey, thank you for this engaging set of posts - and the entire post and thread, really. It really opened my eyes up and made me think hard - exploring a new mental space, in Star Trek speak ( see https://memory-alpha.fandom.com/wiki/All_Good_Things…_(episode)#Act_Ten for the reference - it happens near the end of the episode)
I think I prefer 1 or the ? version. Going with 2 means I can’t interact with a post that fails to federate (or is older than my instance) at all. With 1 or ? I can. Being able to backfill comments would be icing on the cake.
From the linked post of https://discuss.tchncs.de/post/1907501 it seems the standard bearer is just what the home instance of the community has for that post. So comments and votes made to a post after defederation do not count. This makes sense - if you can’t trust the other instance, probably you can’t trust their admins either - and it’s easy for an admin with database access to add fake comments and up/downvotes, etc. So we just pick the obvious source of truth and roll with that.
One small issue here - from screenscraping we can get the vote total count but maybe not the individual votes. We wouldn’t know who voted which way. That’s not a big deal - we can just make up a “random0123456@example.com” ap id to own the vote in the backfilled cases, but I think it makes the case for ? rather than 1.
(As an aside I don’t think Lemmy does backfilling, only pyfedi does it - as the top comment says, “just… it does not backfill previous content and votes.” But the post itself I think must get picked up when a new comment or up/downvote gets seen.)
Damn I loved that Star Trek episode - it was so great! It’s so easy to just keep doing the same old thing - e.g. to explore the stars, but to join a card game - THAT would be a NEW adventure, surely.
But as for the possible, one idea would be to make a hard separation: let’s say a PieFed user joins a new community XYZ. As of that very moment, all newly made posts along with all votes come in, and everything that can make it there through any federation networking issues are reflected in that posts, BUT all old posts do not show up - even if new votes or comments are sent, those are simply discarded - and perhaps a message is displayed “there are no more posts available from this community”. So everything NEW is fully there (to the extent possible/feasible), while everything OLD is fully missing (it would have been only partial/misleading information anyway), and this way there is nothing in-between.
At least, it’s a thought:-). Surely it would not please everyone, but it might be a least worst option?
Err - so basically do what Lemmy does?
Like I said, that wouldn’t work for me. But then again, I’m an admin of my single user instance. And I can see the needs of “real” instances - ones trying to build a community of many users - being different from mine.
So maybe allow it as an admin toggle, default to Lemmy behaviour. That would seemingly cover everyone - and the admins on piefed.social can manually pull in an older post/comment if there’s a compelling reason to make an exception (and add the cautions about possibly inaccurate vote counts and such).
No (I think?) - I thought your link showed that Lemmy was showing inaccurate information as well. So what I was proposing was perhaps better than what Lemmy does, in the sense that Lemmy would pull in the additional posts upon a new voting occurrence, but PieFed could block that and either show only posts for which fully accurate information is available, and then discard those where the extra (voting) information is not available, or else it could go a step further and go ahead and show those posts that only have partial information but make sure to label it specially to indicate that. Lemmy does neither of those.
Second issue: right - an admin should be able to, and maybe there’s no real reason to prevent regular users either, but it shouldn’t be so common, so it would be nice to move the link to do that somewhere else so that new users (like me!) don’t get confused about what it means and is supposed to do and pull things in on a whim without knowing the factors involved.
On the 1st point - well yes, Lemmy isn’t completely accurate either. But that’s inherent to federation. The suggestion is a good one, but consider this:
Showing that only partial info is available is possible, just compare to the home instance’s page via screenscraping and throw up the partial info banner if the counts look different.
Beyond that, discarding the extra comments and partial voting info that could be retrieved via screenscraping is basically what we do now - it’s literally just doing nothing with the extra info. By contrast, doing stuff like figuring out the comment URL or ap_id and stuffing it into a database, or creating dummy actors to store partial voting records would actually require extra effort.
So “1” is impossible, and “?” requires a lot of smart effort, while “2” is the easiest. Usually the impossible things are the ones most worthwhile to build.
On the 2nd point, agreed. Perhaps there is a better place to put that link - could just live somewhere on the Community’s page rather than on each on and every post.