I recently bought myself a new server. It contains a AMD Ryzen 5 3600, 32Gb of RAM harvested from my desktop because prices¹, and a motherboard I'm not particularly happy about. My build has two killer features: 4*4 Tb of storage in RAID-Z1 (Bought before the prices ¹), and 2x10Gb ethernet NIC on a expansion card. And since I have moved away from home, no father of mine will complain about the electricity usage. This is my treat for gaining my first job as a developer for the Swedish Armed Forces.

There are many choices when designing a system. What OS should I use?, Should I host native or in VM, containers, designing for horizontal scaling, cloud… While these are all good questions when designing for production, the great thing with a homelab is that there is no SLA². You are free to design, redesign, and break however you want. Use bleeding edge software or unmaintained software without getting fired!

My previous designs have all been single host setups with limited separation through different types of abstraction of Linux process seperation³. That was partly for the learning experience, but while that design pattern taught me some about Linux, I rather learnt about the abstractions, be that NixOS or docker. More significantly, it was because of hardware restrictions making me wary towards virtualisation and networking.

Now that the hardware restrictions has loosened up, and my curiosity has moved further towards networking, I thought it a good time to install a type-1 hypervisor⁴. My demands for the os is, as always, Free as in Beer and Free as in Freedom leaving surprisingly few options. The commercial options, primarily ESXi, citrix and Hyper-V, is a no-go. The open-source options I found where proxmox, XCG-ng and IncusOS. There also exist Red Hat OpenShift, which seemed too much like a cloud than a hypervisor, and I later on found out about Red Hat oVirt, which I don't know much about.

The most popular of these options is Proxmox. From my personal experience, as well as talking to some friends; the OS is well established, competent, and feature rich. My understanding is that Proxmox is based on debian but with a custom kernel. Recollecting what I read on the trustworthy source of Reddit, proxmox is unreliable because of its weird upstream and chaotic kernel. I know of production proxmox instances and have a hard time believing that. Other Redditers state that proxmox is not a type-1 hypervisor, instead opting to call it a type-1.5 hypervisor. I find their argument reasonable. Since a privileged administrator are provided a root shell, user-land is fully exposed intensionally, and even required to enter if you wish to do trivial administration such as PCIe passthrough. Proxmox has builtin support for LXC and OCI containers, exposing the kernel to clients. Generally accepted type-1 hypervisors, such as ESXi and Hyper-V does not allow this, leading to a more reliable core. I would personally recommend proxmox. It has a rich community, is used in enterprises and has a long history.

XCP-ng is a rather young operating system and type-1 hypervisor. The initial release was 31 mars 2018 as a revival of the XCP project, which Citrix XenServer is a commercial version of. XCP-ng is based on the xen hypervisor backend, compared to proxmox and IncusOS that uses the kvm backend. From my understanding, xen is more tailord toward a true type-1 hypervisor, while kvm is tailord to be used as a type-2 hypervisor, and seem to be the more popular one in the linux sphere, being the one installed on most generall purpose distros. Another popular distro designed around Xen is qubesOS, a reasonably secure desktop operating system designed around virtual machines called qubes. XCP-ng uses the linux kernel and from what I could find is not downstream of any other linux distribution.

IncusOS is a linux distribution based on debian. Since this is the operating system I chose, its the one I know the most about. The OS is very young, I believe the official release date to be 1 Nov 2025 although looking at the source code, the first tag is 202412062204. I choose this operating system because I wanted to. While the OS has been stable and reliable, it's young age is apparent when trying to do something as simple as changing ip address on a NIC. Mostly due to the documentation and lack of community posts. That being said, I'm happy with my choice and would choose it again. I wouldn't recommend it yet over the alternatives for production use, but see a bright future for the novel exciting idea.

The thing about IncusOS is that it is a OS purely centered around incus. IncusOS, together with incus is a project under linuxcontainers.org, the project behind LXC, LXCFS, Distrobuilder and more. Incus is an interface too virtualisation and containerisation technologies. While its primary focus has been LXC containers, it also allow you to interface with oci-containers (docker), and KVM/libvirt. It has full support for configuration of virtualised and physical networking and more. Whats truly special with incus is the seamless cooperation of the hosting technologies. An administrator barely have to care if the machine is virtual, LXC containerised or OCI containerised. The interface is the same, the configuration is as close to same you could realistically get, and they share the exact same networking, allowing for a seamless cooperation between the clients. This allows you to have the quick simplicity and performance of oci-(docker) containers, the versatility and reliability of VMs, and the VM-like experience coupled with container performance of LXC container; without the choice having any broader consequences. While proxmox too has support for LXC and OCI containers, my understanding is that they are more separated, especially in the network layer. XCP-ng as a true type-1 hypervisor does not share the kernel with any clients.

While IncusOS is not a year old, incus is older, being released 7 oct 2023. While that is young compared to competing products, the incus cli tool feels competent and is fairly nice to use. The web-ui has long way to go, and does not have all the functions the cli has. Incus has traditionally been installed alongside a general purpose operating system such as debian, rhel, or suse. The deal with IncusOS is that it is truly ljust made for incus. That does not entail a minimal install of debian with incus slapped on. The only way to interact with the system is through the incus REST api. There is no SSH port, nor can you interact by plugin in a keyboard directly into the server. At no point during the installation do you use the keyboard, instead you inject the mTLS authentication certificate along with other configuration at the download section of the website!

The OS seem to have a high focus on security. It enables secure boot per default, and I don't think you can disable encryption on the ZFS boot drive, which has it's key stored in the TPM. There exists a none-TPM option for users without that hardware, although it appears to be an afterthought. The only means of authentication is mTLS and kerberos. Since I don't have windows Active Directory at home, I choose the default (mandatory?) option of mTLS. The certificate, as mentioned before, is injected when downloading the ISO, and is used to authenticate to the webui as well as the incus cli. The process was a bit yank for the browser section due to the encrypted certificate, and the incus-cli was also a bit difficult to get working. I ended up going the manual route of poking in .config/incus, but you are supposed to do it through the cli. When it was working, it felt like there was no authentication. Incus cli just worked since it found the authentication certificate, and librewolf just asked me if I wanted to provide the certificate.

While the installation for me was sunshine and rainbows (apart from the cpu not having integrated graphics during bios setup), the setup was not. I had a lot of problems configuring my disks. The only way I found to mitigate this was by booting GParted and wiping my drives. Now that I feel more comfortable with the incus cli, I'm pretty sure it was fixable, although that early, I was lost in the incus-cli yaml swamp. I also had a lot of problems configuring the NICs. Again, the functionality was in the incus cli, but I was lost and ended up removing the network configuration leaving every nic empty. Since there is no keyboard interactions with the system, the only way to solve my problem was to reinstall the system.

I have a few gripes with incus. The terraform integration is fairly bad, frequently resolving with obvious bugs; the webui is good enough to be useful, and just enough for you to borrow into a hole before switching to the cli. There are things it does better than anything else. No other system has the same seamless integration of vm, lxc-, and oci-container. No other system has such as great cli. No other system has such a tight integration with LXC-containers.

I still have a long way to go with my homelab. Having a job showed me how much time you have as a student compared to worker: the progress is not as fast as I would have liked. That being said, all the essential stuff is up and running, with an exciting novel operating system I have a blast working with. I think a hypervisor was the right way to go. The ability to design the network, run any virtual operating system, and having many hosts allow me to experiment the way I want to. Incus allow me to run anything I want, while encouraging the use of containers that keeps my hardware usage low. While Incus is young, and IncusOS an infant, I think the product is great and recommend you check it out. For production systems, I think stick to Proxmox or XCP-ng as of yet. Proxmox too have support for LXC and OCI images, and nothing prevents you in XCP-ng to spin up a VM specific for containers - that's what I would recommend for an enterprise solution due to increased security and reliability. While IncusOS is a OS for a very specific niche, you should keep the incus tool in mind. You may want to run it on your desktop for container orchestration, or in a VM at an enterprise. It will only get better!