Incus - Container and Virtual-Machine Manager

Linux Containers Project

Linux
Containers
Published

February 20, 2024

Modified

February 23, 2024

Linux Containers (LXC) 1…

Installation

Setup a test-environment based on Libvirt and Vagrant:

# restart after following configuration
echo "options kvm_intel nested=1" | sudo tee /etc/modprobe.d/kvm.conf

# configure a VM for testing
pushd $(mktemp -d /tmp/$USER-vagrant-XXXXXX) 
# prepare a virtual machine for testing
cat > Vagrantfile <<EOF
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
  config.vm.box = "generic/ubuntu2204"
  config.vm.provider :libvirt do |libvirt|
    libvirt.memory = 4096
    libvirt.cpus = 2
    libvirt.nested = true
    libvirt.cpu_mode = "host-passthrough"
  end
  config.vm.synced_folder ".", "/vagrant", disabled: true
end
EOF
vagrant up && vagrant ssh
# ... work with the VM
vagrant destroy
popd && rm -rf /tmp/incus-$USER-vagrant*

Incus 2 on Debian 12:

mkdir -p /etc/apt/keyrings/
curl -fsSL https://pkgs.zabbly.com/key.asc -o /etc/apt/keyrings/zabbly.asc
cat <<EOF > /etc/apt/sources.list.d/zabbly-incus-stable.sources
Enabled: yes
Types: deb
URIs: https://pkgs.zabbly.com/incus/stable
Suites: bookworm
Components: main
Architectures: amd64
Signed-By: /etc/apt/keyrings/zabbly.asc
EOF
apt-get update && apt-get install -y incus
incus admin init --minimal

Incus can be run as a regular user…

  • …group incus allows basic user access …restricted to a per-user project
  • …GROUP incus-admin allow full control of incus
# add a user to the incus group
sudo usermod --append --groups incus $USER

Instances

Types of instances…

  • Virtual machines …use qemu to provide the VM functionality
    • …use a dedicated kernel …support non Linux
    • …require hardware support …more resources then container
  • System containers …implemented through the use of liblxc (LXC)
    • …in contrast to Docker/Podman application containers
    • …software-only …less resources then virtual machines

Incus uses an image-based workflow…

  • …instance based on an image …available from a remote image store
  • …identified by a fingerprint
  • …images downloaded and locally cached

init, launch, stop & delete

Commands…

  • init …(only) create instance
  • launch …creates and starts instance
  • stop …stop an instance
  • delete …remove instance
# create & start a system container
incus launch images:ubuntu/22.04 [$name]

# …a virtual machine
incus launch images:ubuntu/22.04 [$name] --vm

# …remove a running instance
incus delete -f $name

# prevent accidental deletion of instances
incus alias add delete "delete -i"

Privileged containers… created by root and run as root…

  • …may not be appropriate for production use
  • …may be protected by:
    • …capability dropping
    • …apparmor profiles
    • …selinux context
    • …seccomp policies

list & info

# list all instances
incus list

# list any instance with name 
incus list $name 

# key/value pair referring to a configuration item
incus list status=running

# columns: state, arch., creation date, profile and type
incus list --fast

# abbreviated list of columns to show
incus list -c n4st

info …detailed information about an instance:

incus info $name

# …latest log lines for the instance
incus info --show-log $name

console

Attach to instance consoles …available from boot:

incus console $instance

# …including previous logs
incus console --show-log $instance

# …attach to the console when you start your instance
incus start --console $instance

# …VM console with graphical output
incus console --type vga $vm_instance

exec & file

exec runs commands inside an instance…

  • …either interactively or non-interactively
  • Options:
    • --user …user ID for running the command
    • --cwd …directory in which the command should run
    • --env …pass environment variables to an exec session
# interactive shell (root by default)
incus exec $instance -- /bin/bash
# …as a different user
incus exec $instance -- su --login $user_name

# specify a directory to run a command
incus exec --cwd /tmp $instance -- ls -l

# execute with a given UID (nobody for example)
incus exec --user 65534 suited-bug -- whoami

# set an environment variable
incus exec --env FOO=BAR suited-bug -- env | grep FOO

file command manages files inside an instance …sub-commands:

  • edit an instance file
  • delete a file from instance
  • pull …copy file from instance
  • push …copy file to instance
# file must already exist on the instance
incus file edit $instance/etc/hosts

# remove a file
incus exec $instance -- touch /tmp/foo.txt
incus file delete $instance/tmp/foo.txt

# copy file to instance
echo "foo" > /tmp/bar.txt
incus file push /tmp/bar.txt $instance/tmp/

# print file from  instance to STDOUT 
incus file pull $instance/tmp/bar.txt -

# copy file from instance
incus file pull $instance/tmp/bar.txt /tmp/foo.txt

config

Configure instances…

  • properties …set when the instance is created
  • options 3 …directly related to the instance
# extended information about an instance
incus config show -e $instance

# configuration option…
incus config set $instance limits.memory=128MiB
incus exec $instance -- free -m
# …show configuration option
incus config get $instance limits.memory

# increase rootfs storage capacity
incus config device override $instance root size=30GiB
incus restart $instance
incus exec $instance -- df -h

# remove a configuration option
incus config unset $instance boot.autostart

Images

Pre-configured remote servers 4

# list remote servers
incus remote list

# list all remote images on a server
incus image list $remote:
incus image list -c ld $remote: | less

# …specific images
incus image list $remote:rockylinux architecture=amd64
# …details about images
incus image info $remote:rockylinux/9/amd64

Images contain a root file system and a metadata file 5

  • metadata.yaml information relevant to running the image
  • rootfs/ directory contains a full file system tree
  • templates/ (optional) dynamically create files inside an instance

Two Incus-specific image formats…

  • Unified tarball …single tarball (usually *.tar.xz)
  • Split tarballs …one for metadata & templates …another for the rootfs

Storage

Storage pools…

  • …contain volumes of different types
    • …container/virtual-machine …used for instance
    • …image …storing container or VM images
    • …custom …storage volumes to hold data
  • …uses a storage driver 6
    • …facilitate storage back-end
    • …defaults to directory driver
    • …recommended to us ZFS & Btrfs …shared with the host
    • …remote storage supported with Ceph & LVM cluster
  • …storage location depends on the storage driver

Creating an instance…

  • …automatically creates a storage volume
  • …used as the root disk for the instance
# list storage pools …driver and location
incus storage list

Network

Connect an instances to a network…

  • …either managed network …fully control
    • …network bridge …virtual layer-2 Ethernet switch
    • …virtual software-defined network (SDN) based on OVN 7
  • …or connection to an external network …limited control
    • …typically connection to a physical network interface
    • Macvlan …several IPs on single network interface
    • SR-IOV …single network card port as virtual network interfaces
# list all networks
incus network list

# details about a specified network
incus network show $network

Bridge

Singe system deployments typically run a network bridge 8…

  • …support for native Linux bridges and Open vSwitch
  • …instance share a layer-2 bridge …defaults to a NAT configuration
  • …setup of local dnsmasq for DNS, DHCP (IPv6 route announcement)

Instances assigned to at least on network device…

  • …defaults to the default network bridge
  • …defined by the default profile
# using the default configuration
incus launch images:rockylinux/9 $instance
incus exec $instance -- dnf install -y nmap-ncat
# list on port 1234 in the instance…
incus exec $instance -- nc -l -p 1234
# …get the IP from `incus list` …connect from the host
nc -v $ip 1234

By default instance can not be connect to from outside the host.

Ingress

…expose a service running on an instance to the outside world

Add a dedicated network route to an instance on the network router…

# get the sub-network address (CIDR) of the instance (as a user, not root!)
instance_sub_network=$(sudo incus network list \
    | grep incusbr-$UID | grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}/[0-9]*")
# get the IP address for the node hosting the instance
host_ip_address=$(ip addr show eth0 \
    | grep 'inet ' | grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}" | head -1)
# print the command to run on the network router
echo "ip -4 route add $instance_sub_network via $host_ip_address"

Example

Get an SSH server running…

# create $ launch a new container instance
incus launch images:rockylinux/9 $instance
# install the SSH server package
incus exec $instance -- dnf install -y openssh-server
# write a configuration file to allow root login with password
echo 'PermitRootLogin yes' > rootlogin.conf
incus file push rootlogin.conf \
       $instance/etc/ssh/sshd_config.d/100-rootlogin.conf
# restart the service within the container
incus exec $instance -- systemctl enable --now sshd.service
# set the root password in interactive mode
incus exec $instance -- passwd