With the advances in gaming on Linux in recent years, it is so tempting to switch full time. I would absolutely love to, but I am a Game Pass Ultimate subscriber and it is where I play a lot of my games on PC. I know you can use the cloud version, but I cannot stomach streaming games in their current state, so it is a no go. A large portion of my Steam library is compatible, but anytime I have done an install I end up giving in and going back to Windows for games.
So yes. I had a similar setup to you, passed through my Nvidia card to Windows and kept my onboard Intel card for Linux, but much like you I wanted to game with both Linux and Windows, so now my onboard Intel card is disabled and instead I have some qemu scripts that detach the Nvidia card from Linux and to the VM, and vice versa once the VM is shut down. Was a pain to get setup, but actually works really well.