I also host my own mailserver and I agree that it mostly works fine. However, there are some email providers that cause trouble:

Google seems to randomly sort some of my mails into the recipients spam folder, while others are delivered fine to the respective inbox. It kinda sucks that you can never be sure whether the recipient actually received your mail or whether they just don’t reply. My IP and domain are not blacklisted on any spam list; SPF, DKIM and DMARC are set up correctly as well.

Even worse is the Telekom (German ISP), who use an explicit whitelist of IP addresses (only IPv4 of course) and require you to display your contact information publicly on a website reachable via the same domain your mailserver uses. Once you’ve set this up you need to message them to be put on their whitelist. If you’re not on their whitelist, they simply reject your mails, they are not even delivered to the spam folder (maybe it’s not worse than Google, because you at least get a notice from your mailserver that your mail couldn’t be delivered). In the end I decided that I don’t care enough to comply with their regulations and just don’t send any mails to Telekom customers.

Aside Google and Telekom, I’ve really never had any issues though.