Great news! I started my selfhost journey over a year ago, and I’m finding myself needing better hardware. There’s so many services I want that my NAS can’t handle. And I unfortunately need to add GPU transcoding to my Jellyfin setup.
What’s the best OS for a machine focused on containers and (getting started with) VMs? I’ve heard Proxmox
What CPU specs should I be concerned about?
I’m willing to buy a pre-built as long as its hardware has sufficient longevity.
Why not use Proxmox to host the containers directly instead of using a VM? I know it’s easier to use this way but it kinda misses the point of using proxmox then
Not at all. Proxmox does a great job at hosting VMs and giving a control plane for them - but it does not do containers well. LXCs are a thing, and it hosts those - but never try to do docker in an LXC. (I tried so many different ways and guides and there were just too many caveats, and you end up always essentially giving root access to your containers, so it’s not great anyway). I’d like to see proxmox offer some sort of docker-first approach will it will manage volumes at the proxmox level, but they don’t seem concerned with that, and honestly if you’re doing that then you’re nearing kubernetes anyway.
Which is what I ended up doing - k3s on proxmox VMs. Proxmox handles the instances themselves, spins up a VM on each host to run k3s, and then I run k3s from within there. Same paradigm as the major cloud providers. GKE, AKS, and EKS all run k8s within a VM on their existing compute stack, so this fits right in.
Thanks for sharing your experience ! I was kinda interested for my new N300 if I should install promox+LXC-docker or promox+VM-docker !
Hearing you had a lot of issues and caveats makes my choice easier wihout even giving it a try ! So thanks !
I really wanted it to work, for me it made the most sense I thought, as little virtualization as I could do. VM felt like such a heavy layer in between - but it just wasn’t meant to work that way. You have to essentially run your LXC as root, meaning that it’s essentially just the host anyway so it can run docker. Then when you get down to it, you’ve lost all the benefits of the LXC vs just running docker. Not to mention that anytime there was even am minor update to proxmox something usually broke.
I’m surprised Proxmox hasn’t added straight-up support for containers, either by docker, podman, or even just containerd directly. But, we aren’t it’s target audience either.
I’m glad you can take my years of struggling to find a way to get it to work well and learn from it.
Docker runs fine nested in lxc with uid/gid mapping.
The difficulties of running docker in lxc are particular to proxmox, I ran docker in lxc on proxmox for years, but I’m glad I moved incus; much more sensible approach.
Thanks for your point of view ! I’m still new to proxmox and I went the LXC route… Seems to be working well so far but time will tell!
That’s a great question…redundant ports and all that