Hello everyone! During one of those illuminated evenings, I got the idea to move my small server in Scaleway to some more powerful server in Hetzner. If I will make the move, I am thinking of splitting the server in various VMs, to host different services that belongs to different trust boundaries, for example:
- A Lemmy/writefreely instance
- Vaultwarden/Gitea
- Wireguard tunnel to my home infrastructure
- Blogs, and other convenience services
In order to achieve the best level of separation, I was thinking of using VMs. My default choice would be Proxmox, because I used it in the past, and because I generally trust it, however I am trying to evaluate multiple options, and maybe someone has good or better experiences to share.
Other options I thought about are:
- Run everything in Docker. I am going to do this nevertheless, but Docker escapes are always possible, especially with public facing images that I did not write myself and/or that require a host volume.
- KVM directly? I am OK even without a GUI to be honest. I am not aware if there is some ansible module or even better Terraform provider for this, it would be great.
- ESxi? I have no experience with this solution.
Any idea or recommendation?
I’d go with Proxmox with a docker VM then you can always run other VMS or lxc containers if needed.
Yeah, probably this is the way I will go, to be honest. I just wanted to bounce some ideas in case I was missing out on some other technology, and a few people mentioned some stacks in this threat which are pretty obscure to me, so nice to look into them and compare!
Why rent a whole server? You can run a cloud VM at a fraction of the cost.
If you’re breaking into the industry, I’d say esxi. If it’s a hobby, then proxmox since you’re already familiar.
I use libvirt and never found a reason to switch to something else. Easy to script, easy to manage with the gui
Proxmox has been great for me.
Proxmox is nice, xcp-ng works I suppose, even though its a bit… niche thing 2TB disk limits though
Personally, after looking at what the industry wants; I would start my homelab trying to automate it with Ansible/Terraform.
libvirtshould be decent, and if you want to go over to BSD, I think ansible supportsbhyve? If not,libvirtdefinitely runs on BSD so you could just automate thatI work in security, so there is no really devops/sysadmin prospect for me. That said, I use ansible and (mostly) terraform professionally and for my lab, so that’s a good idea nevertheless. I don’t have much BSD experience, what do you think are the key reasons to go that route instead of Linux?
For me, it’s a personal decision. I find BSD more cohesive. That is subjective and has been debated for a decade now. I also find
bhyvea bit easier to use, albeit the features are newer and more in number in KVM (for example:bhyveuntil very recently didn’t have VirtIO drivers, so Windows machines would be useless on it).I’m interested in working in Security myself. Would you be able to tell me a little more about your work? Also, what role/path in security would you recommend for a Cloud admin/System Admin?
In the places where I’ve had to make similar decisions, I’ve used the need for ‘advanced’ features to make the call. If I’m looking for storage or networking redundancy, or I’ve been interested in running multiple hosts systems, or I’ve been looking to play with overlay networks, then I’ll grab Ovirt, Proxmox, VSphere, or Openstack (depending). When I just want something simple-ish, I just KVM / Podman on a Linux machine.
Good point, I don’t have any advanced use case, except maybe some slightly more complex network setup. Probably this is achievable with KVM too (and/or some firewall-fu). I would like to have fully IaC, so I don’t have to click through guis, so the availability of Terraform providers might be a dealbreaker (which I didn’t look yet for Proxmox, for example).
