If you can play 5 on anything other than the Switch I would do that for performance reasons. Otherwise I think it really depends on whether you want a top-down view or a 3rd person 3D one. If you are okay with top-down, go with RF4. If that does not appeal to you at all, RF5 is still serviceable on the Switch, but watch some videos to make sure you’re okay with the performance.
If same sex relationships are important to you, RF5 is the only one that allows it. Otherwise just pick the one that’s more visually appealing to you. If you go with RF5 first, you won’t know what you’re missing from RF4 and can still go back and play it at some point in the future.
Speaking of, RF4 is also on Xbox and PC Gamepass, as well as PS+