Skip to content

Home

Jump to bottom
Bradley Leonard edited this page Nov 13, 2024 · 3 revisions

Welcome to the proxmox-qdevice wiki!

History of the Project

With other priorities in life taking precidence, I found myself with limited funds to build out my homelab. I've more or less ended up with two (2) physical servers to play around with. I tend to use one (1) for any work related stuff and the other for non-worked related stuff.

After playing with OpenStack for a while, while it met my needs, it was overly complicated for home use (for me at least). A frend suggested ProxMox and it met the need. Been running it ever since. While I'm not a long time user, I did start with version 6.

Normally running an even number of servers in a cluster isn't a problem, but I've had situations where I've booted both promox servers at the same time. In that case, the first server to come online doesn't have a quarum (1 of 2) so none of the virtual images won't start. The 2nd server will (2 of 2). For the most part, what ended up happening is that my work server was the server that didn't have quarum, so many if not all of the virtual images didn't come up. To be honest, its not a problem until i needed the work images. To make a long story short, work run ons on 21 virtual instances. A complete boot of the entire environment takes about 30 minuts from start to a usable environment I can use with errors.

So I needed a solution.

With an external qdevice thats already up, the first device to come up has quarom (2 of 3). This means I can reboot, walk away and not worry about lack of quarum when restarting.

Pre-requisites

The pre-requistes are pretty simple:

  • Docker
  • Docker Compose (not really necessary, but I include one to assist you or you use with portainer)
  • Internet Connection (again not really necessary, but i do have a docker image pre-built so you can use that if you want to)
  • A proxmox cluster with an even number of servers - If you have an odd number, you don't need this solution.
  • A dedicated IP address in your network to bind the container too - This is required for THIS solution. There are solutions that change the SSHD port (22) to another port. This breaks the Proxmox requirements. While its probably fine and will work, Proxmox could make a change that would invalidate solutions that run SSHD on a port different than 22.

What's Supported?

The container as designed will run on the following:

  • x86-64 - for the most part any OS that runs on 64 bit Intel compatible hardware that can run docker. There are some exceptions, so see below.

What's NOT Supported?

As more and more pople use this, we've gotten reports that this solution doesn't work or has issues.

  • Synology - Please see Issue #10. This appears to be an issue with the kernel. See Issue #2. The work around would to build your own container based on Debian 11 (Bullseye).
  • QNAP (ARM based) - Please see Issue #1. It appears that somebody has built a container to run on a ARM-based QNAP server. See here.
  • Running SSHD on a port other than 22 - Promox requires a qdevice running on port 22. See Issue #3.

What's My Configuration?

I have a two (2) node Proxmox cluster. The two nodes are not running a HA configuration, but rather one (1) node contains all of my virtual instances I use for work (plus a few others) and the second node is for all my personal virtual instances.

I needed an external qdevice running on a device that wasn't one of the two. I have a Network Attached Storage (NAS) device that has the ability to run virtual machines. I have various raspberry pi's running as well, but my NAS runs 24x7 with a UPS so for availablity and stability, I went with it.

I created a small virtual instannce on the NAS server with 1 CPU, 1 GB memory & 10 GB disk space. It auto starts on boot. The base OS runs Debian (latest version) and has docker installed on it. I've attached two virtual ethernet adapters to the virtual instance. The first is for the virtual instance and the 2nd is what I use to attached to docker.

I created a MACVLAN network (see the docker compose file) that has the range of 1 IP address. The MACVLAN network is then attached to the container when it starts. The container then binds to the MACVLAN network and assigns a hostname and static IP Adddress to the container.

In my DNS server, I've created an entry for the specific Hostname / IP Address combination. This way I keep track of the IP Address assigned and I can reference the container by hostname if needed.

In my proxmox configuration, I use the IP address assigned to add the qdevice.

Clone this wiki locally