Docker — Community Edition (CE)

Published

January 31, 2025

Modified

February 28, 2025

Docker Community Edition (Docker CE) — Free and open-source version of Docker

Installation

Install Docker from the official package repository2:

# Enterprise Linux 9
sudo dnf remove -y podman-docker
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf install -y git docker-ce docker-ce-cli containerd.io docker-compose-plugin
sudo systemctl enable --now docker
sudo usermod -aG docker $USER

# Fedora >=41
sudo dnf remove -y podman-docker
sudo dnf config-manager addrepo --from-repofile=https://download.docker.com/linux/fedora/docker-ce.repo
sudo dnf install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
sudo systemctl enable --now docker
sudo usermod -aG docker $USER

Configuration

Configure Docker…

  • …CLI in ~/.docker/config.json …environment variable DOCKER_CONFIG=path/to/dir
  • …daemon in /etc/docker/daemon.json …or Systemd drop-in

HTTP Proxy

Docker daemon:

mkdir -p /etc/systemd/system/docker.service.d
cat > /etc/systemd/system/docker.service.d/http-proxy.conf <<EOF
[Service]
Environment="HTTPS_PROXY=$HTTPS_PROXY"
EOF
systemctl daemon-reload && systemctl restart docker.service

Docker client:

# building a container
docker build --build-arg HTTPS_PROXY="$HTTPS_PROXY" .

# executing a container
docker run --env HTTPS_PROXY="$HTTPS_PROXY" #…

…persistent…

# proxy configurations for the Docker client 
cat > ~/.docker/config.json <<EOF
{
 "proxies": {
   "default": {
     "httpProxy": "$HTTP_PROXY",
     "httpsProxy": "$HTTPS_PROXY",
     "noProxy": "$NO_PROXY"
   }
 }
<}
EOF

# check functionality by printing the environment variables in a container instance
docker run --rm alpine sh -c 'env | grep -i  _PROXY'

Usage

Docker commands to work with container images:

docker pull $image      # download container from registry
docker images           # list containers on local system
docker rmi $image       # remove a container from local system

Work with Docker container instances:

Creadocker ps               # list running containers
docker stop $container  # stop running container
docker rm $container    # remove container instance

# ephemeral interactive container instance
docker run -it --rm rockylinux:9

Buildx

docker buildx ls                 # list builders
docker buildx inspect $name      # print details on a builder
docker buildx rm $name           # remove a builder
docker buildx use $name          # set builder as default

docker buildx — Use BuildKit3 to build container images…

  • …improved back-end to replace the legacy builder
  • …default for Docker Engine >=23.0
  • Features…
    • …parallelize multi-stage builds with custom contexts
    • …build images for different platforms/architectures

Create

# create a new build instance
docker buildx create $options --name $name #…

Common options for buildx create

  • --driver — Configure where the BuildKit back-end runs…
    • docker (default) BuildKit library bundled into the Docker daemon
    • docker-container dedicated BuildKit container using Docker
    • kubernetes creates BuildKit pods in a Kubernetes cluster
    • remote connects directly to a manually managed BuildKit daemon
  • --driver-opt for the docker-container driver4
    • network=host host network mode for the container
    • image=moby/buildkit:latest5 sets the image to use for the container

Bake

docker buildx bake          # build all targets
docker buildx bake $target  # build a specific target

Bake file6 — Build images using a declarative approach

  • …define your build configuration & build arguments
  • …in a file typical docker-bake.hcl7 or docker-compose.yml
  • Property types…
    • target specifies a build target
    • group defines a collections of build targets
    • variable sets build arguments
    • function defines a custome Bake function
docker-bake.hcl
variable "APP_VERSION" { default = "24.11.1" }

variable "REGISTRY" {
    default = "containers.example.org"
}

target "default" {
  matrix = {
    stage = [
      "slurmctld",
      "slurmdbd",
      "slurmd",
      "slurmrestd",
      "sackd"
    ]
  }
  context    = "."
  dockerfile = "Dockerfile"
  output     = ["type=registry"]
  name       = "${stage}"
  target     = stage
  tags       = [ "${REGISTRY}/slurm/${stage}:${APP_VERSION}" ]
}

Clean Up

docker system df                        # check disk usage of all resources

docker stop $(docker ps -aq)            # stop all running containers
docker rm $(docker ps -aq)              # remove all containers
docker rmi $(docker images -q)          # remove all container images
docker volume rm $(docker volume ls -q) # remove all volumes

docker container prune                  # remove all stopped containers
docker image prune                      # remove dangling images
docker image prune -a                   # remove all images (not just dangling)
docker network prun                     # remove unused networks
docker volume prune                     # remove unused volumes

# removes unused data, including stopped containers
# unused networks, and dangling images
docker system prune --volumes
  • Dangling images — Images not tagged and not referenced by any container
  • Option -f — Forcefully remove containers/images even if running/used

Dockerfile

How to minimize container image size?

  • Use a minimal base image
    • Distroless8 does not include a Linux distribution environment
    • Alpine9 Linux distribution specifically for containers
    • Minify container images with extra tools: slim10:
  • Minimize dependencies your application requires (libraries/packages)
  • Multi-stage builds

Build Context

Build context…

  • …typically the directory where the Dockerfile is located
  • …includes all files and directories referenced in the Dockerfile
  • …entire build context send to the Docker daemon (impacts build performance)
# build in the current working directory
docker build -t $app:$app_version .

.dockerignore file — Exclude unnecessary files and directories

  • …minimizes the number of files in the build context
  • …avoid large files, temporary files, or development artifacts
  • …organize the directory structure to streamline the build

Layer Cache

Container images are built in layers …explore with dive11:

  • …each layer represents a set of file-system changes
  • …once a layer is generated it becomes immutable
  • …each instruction in a Dockerfile creates a new layer
  • …each layer is cached …reused when possible to speed up build
  • Layer dependency…
    • …each layer depends on the layers that precede it
    • …layer invalidation forces a rebuild if all subsequent layers
    • …changes to early layers have a cascading effect
  • Cache invalidation — Determines when one or more layers need rebuild
    • COPY or ADD …lines that copy files into an image
    • RUN and other instruction that modify the file-system

Best practices for cache optimization…

  • Maximize cache hits — Order instruction wisely
    • …ordering your Dockerfile instructions strategically
    • …place less frequently changed instructions at the top
    • …use specific versions of dependencies
  • Layer squashing — Combine instructions to reduce invalidation
    • …reduce the number of layers to minimize image size
    • …for example combine multiple commands into a singe RUN instruction
  • Deleting files doesn’t remove them from earlier layers
    • …marked as “not accessible” …still consume space
    • …example: package install & clean up require a single RUN instruction

Multi-Stage

Multiple FROM statements — Define multiple stages in your Dockerfile

  • Why use multi-stage builds?12
    • Separate the build environment from the runtime environment
    • Significantly reduce the size of a final container image
    • Improve security by excluding unnecessary build tools and dependencies
  • Each stage can use a different base…
    • …by default stages aren’t named
    • AS keyword — Give a stage a name…
      • …allows reference to it in a later stage
      • …the base of a FROM statement can be a previous stage
    • COPY statement — Copy files from one stage to another
      • …option --from= used to identify a specific stage
      • …either by name or index (starts with zero)
FROM rockylinux:9 AS build
# …install build dependencies, build software/packages

FROM rockylinux:9 AS base
# copy artifactes from the build stage (packages in this example) 
COPY --from=build /root/rpmbuild/RPMS/**/*.rpm /tmp

FROM base AS #…
#…install & configure application

Build a specific stage with docker build with option --target:

# use tags to distinguish build stages
docker build --target build -t $app:build .

# build final stage ready for production
docker build --target release -t $app:$app_version -t $app:latest .