Incus - Container and Virtual-Machine Manager
Linux Containers Project
Linux Containers (LXC) 1…
- Lightweight OS-level virtualization
- Initially developed by IBM …started in 2008
- …Canonical took over stewardship of the project
- …developed in parallel with the effort to add namespaces to the kernel
- …containers created using a combination of control groups and namespaces
- Either privileged or unprivileged configurations
- …mapping of root UID within the container to a non-root UID on the host
- …tries by default to launch an init system inside of the container
- …data in every container instance is persistent
Installation
Setup a test-environment based on Libvirt and Vagrant:
# restart after following configuration
echo "options kvm_intel nested=1" | sudo tee /etc/modprobe.d/kvm.conf
# configure a VM for testing
pushd $(mktemp -d /tmp/$USER-vagrant-XXXXXX)
# prepare a virtual machine for testing
cat > Vagrantfile <<EOF
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
config.vm.box = "generic/ubuntu2204"
config.vm.provider :libvirt do |libvirt|
libvirt.memory = 4096
libvirt.cpus = 2
libvirt.nested = true
libvirt.cpu_mode = "host-passthrough"
end
config.vm.synced_folder ".", "/vagrant", disabled: true
end
EOF
vagrant up && vagrant ssh
# ... work with the VM
vagrant destroy
popd && rm -rf /tmp/incus-$USER-vagrant*
Incus 2 on Debian 12:
mkdir -p /etc/apt/keyrings/
curl -fsSL https://pkgs.zabbly.com/key.asc -o /etc/apt/keyrings/zabbly.asc
cat <<EOF > /etc/apt/sources.list.d/zabbly-incus-stable.sources
Enabled: yes
Types: deb
URIs: https://pkgs.zabbly.com/incus/stable
Suites: bookworm
Components: main
Architectures: amd64
Signed-By: /etc/apt/keyrings/zabbly.asc
EOF
apt-get update && apt-get install -y incus
incus admin init --minimal
Incus can be run as a regular user…
- …group
incus
allows basic user access …restricted to a per-user project - …GROUP
incus-admin
allow full control of incus
# add a user to the incus group
sudo usermod --append --groups incus $USER
Instances
Types of instances…
- Virtual machines …use
qemu
to provide the VM functionality- …use a dedicated kernel …support non Linux
- …require hardware support …more resources then container
- System containers …implemented through the use of
liblxc
(LXC)- …in contrast to Docker/Podman application containers
- …software-only …less resources then virtual machines
Incus uses an image-based workflow…
- …instance based on an image …available from a remote image store
- …identified by a fingerprint
- …images downloaded and locally cached
init
, launch
, stop
& delete
Commands…
init
…(only) create instancelaunch
…creates and starts instancestop
…stop an instancedelete
…remove instance
# create & start a system container
incus launch images:ubuntu/22.04 [$name]
# …a virtual machine
incus launch images:ubuntu/22.04 [$name] --vm
# …remove a running instance
incus delete -f $name
# prevent accidental deletion of instances
incus alias add delete "delete -i"
Privileged containers… created by root and run as root…
- …may not be appropriate for production use
- …may be protected by:
- …capability dropping
- …apparmor profiles
- …selinux context
- …seccomp policies
list
& info
# list all instances
incus list
# list any instance with name
incus list $name
# key/value pair referring to a configuration item
incus list status=running
# columns: state, arch., creation date, profile and type
incus list --fast
# abbreviated list of columns to show
incus list -c n4st
info
…detailed information about an instance:
incus info $name
# …latest log lines for the instance
incus info --show-log $name
console
Attach to instance consoles …available from boot:
incus console $instance
# …including previous logs
incus console --show-log $instance
# …attach to the console when you start your instance
incus start --console $instance
# …VM console with graphical output
incus console --type vga $vm_instance
exec
& file
exec
runs commands inside an instance…
- …either interactively or non-interactively
- Options:
--user
…user ID for running the command--cwd
…directory in which the command should run--env
…pass environment variables to an exec session
# interactive shell (root by default)
incus exec $instance -- /bin/bash
# …as a different user
incus exec $instance -- su --login $user_name
# specify a directory to run a command
incus exec --cwd /tmp $instance -- ls -l
# execute with a given UID (nobody for example)
incus exec --user 65534 suited-bug -- whoami
# set an environment variable
incus exec --env FOO=BAR suited-bug -- env | grep FOO
file
command manages files inside an instance …sub-commands:
edit
an instance filedelete
a file from instancepull
…copy file from instancepush
…copy file to instance
# file must already exist on the instance
incus file edit $instance/etc/hosts
# remove a file
incus exec $instance -- touch /tmp/foo.txt
incus file delete $instance/tmp/foo.txt
# copy file to instance
echo "foo" > /tmp/bar.txt
incus file push /tmp/bar.txt $instance/tmp/
# print file from instance to STDOUT
incus file pull $instance/tmp/bar.txt -
# copy file from instance
incus file pull $instance/tmp/bar.txt /tmp/foo.txt
config
Configure instances…
- properties …set when the instance is created
- options 3 …directly related to the instance
# extended information about an instance
incus config show -e $instance
# configuration option…
incus config set $instance limits.memory=128MiB
incus exec $instance -- free -m
# …show configuration option
incus config get $instance limits.memory
# increase rootfs storage capacity
incus config device override $instance root size=30GiB
incus restart $instance
incus exec $instance -- df -h
# remove a configuration option
incus config unset $instance boot.autostart
Images
Pre-configured remote servers 4
# list remote servers
incus remote list
# list all remote images on a server
incus image list $remote:
incus image list -c ld $remote: | less
# …specific images
incus image list $remote:rockylinux architecture=amd64
# …details about images
incus image info $remote:rockylinux/9/amd64
Images contain a root file system and a metadata file 5
metadata.yaml
information relevant to running the imagerootfs/
directory contains a full file system treetemplates/
(optional) dynamically create files inside an instance
Two Incus-specific image formats…
- Unified tarball …single tarball (usually
*.tar.xz
) - Split tarballs …one for metadata & templates …another for the rootfs
Storage
Storage pools…
- …contain volumes of different types
- …container/virtual-machine …used for instance
- …image …storing container or VM images
- …custom …storage volumes to hold data
- …uses a storage driver 6
- …facilitate storage back-end
- …defaults to directory driver
- …recommended to us ZFS & Btrfs …shared with the host
- …remote storage supported with Ceph & LVM cluster
- …storage location depends on the storage driver
Creating an instance…
- …automatically creates a storage volume
- …used as the root disk for the instance
# list storage pools …driver and location
incus storage list
Network
Connect an instances to a network…
- …either managed network …fully control
- …network bridge …virtual layer-2 Ethernet switch
- …virtual software-defined network (SDN) based on OVN 7
- …or connection to an external network …limited control
- …typically connection to a physical network interface
- Macvlan …several IPs on single network interface
- SR-IOV …single network card port as virtual network interfaces
# list all networks
incus network list
# details about a specified network
incus network show $network
Bridge
Singe system deployments typically run a network bridge 8…
- …support for native Linux bridges and Open vSwitch
- …instance share a layer-2 bridge …defaults to a NAT configuration
- …setup of local
dnsmasq
for DNS, DHCP (IPv6 route announcement)
Instances assigned to at least on network device…
- …defaults to the
default
network bridge - …defined by the
default
profile
# using the default configuration
incus launch images:rockylinux/9 $instance
incus exec $instance -- dnf install -y nmap-ncat
# list on port 1234 in the instance…
incus exec $instance -- nc -l -p 1234
# …get the IP from `incus list` …connect from the host
nc -v $ip 1234
By default instance can not be connect to from outside the host.
Ingress
…expose a service running on an instance to the outside world
Add a dedicated network route to an instance on the network router…
# get the sub-network address (CIDR) of the instance (as a user, not root!)
instance_sub_network=$(sudo incus network list \
| grep incusbr-$UID | grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}/[0-9]*")
# get the IP address for the node hosting the instance
host_ip_address=$(ip addr show eth0 \
| grep 'inet ' | grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}" | head -1)
# print the command to run on the network router
echo "ip -4 route add $instance_sub_network via $host_ip_address"
Example
Get an SSH server running…
# create $ launch a new container instance
incus launch images:rockylinux/9 $instance
# install the SSH server package
incus exec $instance -- dnf install -y openssh-server
# write a configuration file to allow root login with password
echo 'PermitRootLogin yes' > rootlogin.conf
incus file push rootlogin.conf \
$instance/etc/ssh/sshd_config.d/100-rootlogin.conf
# restart the service within the container
incus exec $instance -- systemctl enable --now sshd.service
# set the root password in interactive mode
incus exec $instance -- passwd
Footnotes
Linux Containers Project
https://linuxcontainers.org
https://github.com/lxc
https://discuss.linuxcontainers.org↩︎Incus Installation, Linux Containers Documentation
https://linuxcontainers.org/incus/docs/main/installing
https://github.com/zabbly/incus↩︎Instance Options, Incus Documentation
https://linuxcontainers.org/incus/docs/main/reference/instance_options↩︎Image Server, Linux Containers Project
https://images.linuxcontainers.org↩︎Image format, Incus documentation
https://linuxcontainers.org/incus/docs/main/reference/image_format↩︎Storage Drivers, Incus Documentation
https://linuxcontainers.org/incus/docs/main/reference/storage_drivers↩︎Open Virtual Network (OVN)
https://www.ovn.org↩︎Bridge Network, Incus Documentation
https://linuxcontainers.org/incus/docs/main/reference/network_bridge↩︎