Skip to content

4.0 Running the Image

Jump to bottom
Rob Ballantyne edited this page Apr 1, 2024 · 5 revisions

Run Locally

A 'feature-complete' docker-compose.yaml file is included for your convenience. All features of the image are included - Simply edit the environment variables in .env, save and then type docker compose up.

If you prefer to use the standard docker run syntax, the command to pass is init.sh.

Run in the Cloud

This image should be compatible with any GPU cloud platform. You simply need to pass environment variables at runtime.

Note

Please raise an issue on the relevant repository if your provider cannot run the image.

Container Cloud

Container providers don't give you access to the docker host but are quick and easy to set up. They are often inexpensive when compared to a full VM or bare metal solution.

All images built for ai-dock are tested for compatibility with both vast.ai and runpod.io.

Template links for both Vast (preferred) and RunPod will be made available in the README.md of the relevant image repository.

Warning

Container cloud providers may offer both 'community' and 'secure' versions of their cloud. If your use case involves storing sensitive information (eg. API keys, auth tokens) then you should always choose the secure option.

VM Cloud

Running docker images on a virtual machine/bare metal server is much like running locally.

You'll need to:

  • Configure your server
  • Set up docker
  • Clone this repository
  • Edit .envand docker-compose.yml
  • Run docker compose up

Compatible VM Providers

Images that do not require a GPU will run anywhere - Use an image tagged :*-cpu-xx.xx

Where a GPU is required you will need either :*cuda* or :*rocm* depending on the underlying hardware.

A curated list of VM providers currently offering GPU instances:

Provisioning Script

It can be useful to perform certain actions when starting a container, such as creating directories and downloading files.

You can use the environment variable PROVISIONING_SCRIPT to specify the URL of a script you'd like to run.

The URL must point to a plain text file - GitHub Gists/Pastebin (raw) are suitable options.

If you are running locally you may instead opt to mount a script at /opt/ai-dock/bin/provisioning.sh.

Each application repository will provide a default script located at https://raw.githubusercontent.com/ai-dock/[repository-name]/config/provisioning/default.sh. This file will be downloaded at runtime unless you have specified your own URL as detailed above. You may use the default file as inspiration for your own provisioning script.

Note

If configured, sshd, caddy, cloudflared, serviceportal, storagemonitor, syncthing & logtail will be launched before provisioning; Any other processes will launch after.

Warning

Only use scripts that you trust and which cannot be changed without your consent.

Clone this wiki locally