You don’t need a desktop environment and can install it later. But it depends on what the distribution offers. There are distributions like Arch which doesn’t have any desktop environment pre-installed. The user is responsible for everything. You can just install a window manager and some other stuff that is needed to have a functional operating system and may never install a desktop environment.

But that requires a bit of time, skill and knowledge of what you are doing. I recommend you to try this out in a virtual machine and to play around without destroying your main computer.