RPM Packages & Yum Package Repositories

Linux

Published

August 12, 2009

Modified

August 27, 2024

Overview

The name RPM ¹ refers to the .rpm file format and rpm program itself:

Binary RPM packages contain compiled software for a specific architecture
- Bundles libraries, executables, data, and configuration files required
- Includes scripts to deploy and integrate the component…
- …configuration generators, start and stop system services, etc
Source RPM packages.src.rpm contain source code to build binary package

Packages use following convention for versioning:

name
name.arch
name-[epoch:]version
name-[epoch:]version-release
name-[epoch:]version-release.arch

Search for RPM packages ² in the repositories of the Fedora Project.

`rpm` Command

rpm query command …-q/--query aliased to rpmquery

rpm -qi PACKAGE_NAME       # general information about a package
rpm -ql PACKAGE_NAME       # list file in an installed package
rpm -qR PACKAGE_NAME       # package dependencies and recommendations
rpm -qf FILE               # query package owning FILE

Verify

# Verify all installed packages 
rpm -Va

# Verify a specific package, verbose
rpm -Vv $package

Using RPM to Verify Installed Packages:

Check if system accurately reflects the contents of the RPM database
Don’t be too surprised if rpm -Va turns up a surprising number of files that failed verification…
- …some attributes change during normal system operation
- …system configuration different compared to builders environment

What Does it Verify? Output is a string of 9 characters, a possible attribute marker

Output	Description
`S`	Size
`M`	Mode …file permissions
`5`	MD5 Checksum …check if all files it manipulates
`D`	Major Number & Minor Number
`L`	Symbolic Link String …is the file’s symbolic link contents
`U`	Owner
`G`	Group
`T`	Modification Time
`P`	Capabilities differ

Attribute flags will only appear if there is a problem…
…otherwise . (period) means the test passed
missing file expected by a package

Attribute markers (left to the filename)

Marker	Description
`a`	Build side-effect file (such as buildid links)
`c`	Configuration file
`d`	Documentation file
`g`	Ghost file (i.e. the file contents are not included in the package payload)
`l`	License file
`m`	File missing, is not a verify failure
`r`	Readme file

`rpmdb`

In case procedures fail due to a corrupted rpmdb files…

error: rpmdb: BDB0113 Thread/process 1394101/139863873272704 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db5 -  (-30973)
error: cannot open Packages database in /var/lib/rpm
Error: Error: rpmdb open failed

…try rpm --rebuilddb

RPM Package Build

Install development tools

# RPM and build tools...
sudo dnf install -y @rpm-development-tools

# Fedora tools...
sudo dnf install git fedora-packager fedora-review

# become member of the mock group
sudo usermod -a -G mock $USER ; newgrp mock

DO NOT build packages as root!

Directories ~/rpmbuild/{SPECS,SOURCES,BUILD,RPMS,SRPMS}

SPECS RPM .spec files…
SOURCES source code archives of target software
RPMS binary RPM packages
SRPMS source RPMs (SRPMs)
BUILD used during the RPM build process…temporary storage

rpmdev-setuptree …set up a SRPM build environment

RPM Spec

# create a new skeleton spec file
rpmdev-newspec ${name}.spec

The “Recipe” for creating an RPM package is a .spec file ³:

Contain the package name, version, RPM revision number
- …steps to build, install, and clean a package
- …a changelog to communicate development details
Multiple packages can be built from a single RPM spec file

Contents of a .spec file ⁴:

Preamble …series of metadata items
Body contains the main part of the instructions…sections…
- %description full description of the software packaged
- %prep commands to prepare the software to be built
- %build commands for actually building the software into machine code
- %check commands to test the software
- %changelog record of changes that have happened to the package

Preamble

Example:

Name:           prometheus-slurm-textfile
Version:        2.0^202404070716
Release:        el8
BuildArch:      noarch

Includes following…

Name …no white space …may include -
Version of the package
- …alphanumeric characters 1.1
- …optionally segmented …handle pre- & post-releases
  - ~ force sorting lower than base …example 1.1~201601 < 1.1
  - ^ force sorting higher than base …example 1.1^201601 > 1.1
- Epoch …override normal version-release sorting
Release …distinguishing between different builds …typically the release version of the distribution
BuildArch …architecture of the resulting binary
- …x86_64 typical Intel & AMD processors
- …noarch …architecture independent

`%install`

Copying the desired build artifacts…

…from %builddir (where the build happens)
…to %buildroot (directory structure with the files to be packaged)

`%files`

List of files that will be installed from the package

Macros in the %files section…

%license …identifies the file listed as a LICENSE file
%doc …file listed as documentation

`rpmbuild`

Build package on the host platform…options…

-bb build a binary package
-bs build a source package
-ba build a binary & source package

# copy input artifacts if required...
cp -vr ./* ~/rpmbuild/BUILD
# build a package from a RPM spec
rpmbuild -ba $package.spec
# list all package in the build directory
ls ~/rpmbuild/**/**/*.rpm

Macros

Evaluate arbitrary strings containing macros on the command line…

rpm --eval "%{_arch}"

# ...provide additional (or overwrite) macros
rpm --define "_release el8" --eval "%{_release}"

# ...display all build-in macros
rpm --showrc

Lists macros which are widely used…

%{_sysconfdir} …defaults to /etc
%{_bindir} …defaults to /usr/bin
%{_sbindir} …defaults to /usr/sbin
…for the build…
- %{_topdir} …%{getenv:HOME}/rpmbuild
- %{_buildrootdir} …%{_topdir}/BUILDROOT
- %{buildroot} …%{_buildrootdir}/%{name}-%{version}-%{release}.%{_arch}
- %{_builddir} …%{_topdir}/BUILD

Override the distribution macros in the ~/.rpmmacros

%{optflags} macro contains flags that determine CFLAGS, CXXFLAGS, FFLAGS…

# ...default build flags
rpm --eval "%{optflags}"

# ...value of the LDFLAGS environment variable set by build systems
rpm -E "%{build_ldflags}"

Consume a macro within the RPM Spec file…

# ...
Version:        1.0
Release:        %{_release}
BuildArch:      noarch
# ...

Use the command-line option --define to pass macro with value…

rpmbuild --define='_release el8' #...

Inspect

Inspect package content…

# show package content
rpm -qipl $package.rpm
# list file in package
rpm2cpio $package.rpm | cpio -t
# extract a single file
rpm2cpio $package.rpm | cpio -idv ./$path/$file

Mock

Mock ⁵ buils RPM packages for different architectures/platforms…

…creates chroots and builds packages within…
…“pristine build environment”, decoupled from the host using containers
mockchain…build chains of packages that depend on each other

Mock target platform configurations…

# for example RockyLinux
find /etc/mock -name 'rocky*'  
# ..or
mock --list-chroots

Build from a source package

mock -r fedora-36-x86_64 $package.src.rpm

Manual Customisation

Modifying a chroot…

# ...install packaghe into a chroot
mock -r $mock_config --install $package_name # or /path/to/rpm
# ...--no-clean option is necessary...
# ...since by default the chroot is cleaned prior to building
mock -r $mock_config --no-clean /path/to/srpm

# copy file between host anc chroot
mock -r $mock_config --copyin /path/on/host /path/in/chroot
mock -r $mock_config --copyout /path/in/chroot /path/on/host

Run commands in the chroot …user -- separator

# ...start an interactive shell
mock -r $mock_config --shell
# ...execute a command
mock --shell -- ls -l '*'

Build Options

Pass build configuration options to rpmbuild…

# ...enable a build option
mock ... --with=selinux ...
# ...disable a build option
mock ... --without=pam ...

Both options above can be used multiple times
Pass arbitral additional options with --rpmbuild-opts
Specify an RPM macro file with --macro-file

Debugging

Check the build.log to determine issues…

…use option --no-cleanup-after to preserve the build environment in the chroot
--clean to purge the chroot tree
--no-clean Do not clean chroot before building package

FedPkg

Clone the Fedora package configuration:

# checkout the Slurm packages without identification
fedpkg clone --anonymous $package
# build the package
fedpkg mockbuild
# list the newly build packages (latest Fedora...Rawhide)
find results_$package/ -name '*.rpm'

Using a specific branch…

# ...switching to the EPEL 8 branch beforehand
fedpkg switch-branch epel8 # List file with changes since epel8
git diff --name-only epel8..rawhide
# List changes since epel8
git diff epel8..rawhide

Target a specific platform…

fedpkg mockbuild --root rocky+epel-8-x86_64

Local Repositories

A repository is:

A collection of packages.
A set of metadata files (the information present in the RPM file). Allows the package manager to operate with the repository without having stored all packages locally.
RPM package files are retrieved on demand at installation time.

Local Upstream Mirror

Simple HTTP server setup:

yum -y install httpd
rm /etc/httpd/conf.d/welcome.conf
systemctl enable --now httpd
# grant access to the HTTP port, or disable the firewall 
firewall-cmd --permanent --add-service=http && firewall-cmd --reload
systemctl stop firewalld && systemctl disable firewalld
# disable SELinux
>>> grep ^SELINUX= /etc/selinux/config
SELINUX=disabled
>>> setenforce 0 && sestatus

Utilities to install:

reposync - used to synchronize remote yum repositories to a local directory, using yum to retrieve the packages
createrepo - Create repomd (xml-rpm-metadata) repository

dnf -y install yum-utils createrepo

Synchronize CentOS YUM repositories to the local directories

mkdir -p /var/www/html/centos/{base,updates}
reposync --downloadcomps \
         --plugins \
         --gpgcheck \
         --download-metadata \
         --repoid=base \
         --repoid=updates \
         --download_path=/var/www/html/centos
# create a new repodata for the local repositories
for repo in base updates
do
        createrepo --verbose\
                   --update \
                   --groupfile comps.xml \
                   /var/www/html/centos/$repo 
done

Create a systemd service unit [2] to execute reposync and createrep:

cat > /etc/systemd/system/reposync-centos.service <<EOF
[Unit]
Description=Mirror package repository
[Service]
ExecStart=/usr/bin/reposync -gml --download-metadata -r base -r updates -p /var/www/html/centos
ExecStartPost=/usr/bin/createrepo -v --update /var/www/html/centos/base -g comps.xml
ExecStartPost=/usr/bin/createrepo -v --update /var/www/html/centos/updates
Type=oneshot
EOF

Use a systemd timer unit [3] to periodically execute the service above:

cat > /etc/systemd/system/reposync-centos.timer <<EOF
[Unit]
Description=Periodically execute package mirror sync
[Timer]
OnStartupSec=300s
OnUnitInactiveSec=2h
[Install]
WantedBy=multi-user.target
EOF

Full Mirror Sync

Full mirror (of a file-system) including ISO images and network install:

mkdir /var/www/html/centos
rsync --verbose \
      --archive \
      --compress \
      --delete \
      rsync://linuxsoft.cern.ch/centos \
      /var/www/html/centos

Note: You can use any CentOS mirror with an rsync:// endpoint.

Create a systemd service unit to execute rsync:

# write a unit file to execute rsync
cat > /etc/systemd/system/rsync-centos-mirror.service <<EOF
[Unit]
Description=Rsync CentOS Mirror
[Service]
ExecStartPre=-/usr/bin/mkdir -p /var/www/html/centos
ExecStart=/usr/bin/rsync -avz --delete rsync://linuxsoft.cern.ch/centos /var/www/html/centos
Type=oneshot
EOF
# load the configuration
systemctl daemon-reload
# start rsync
systemctl start rsync-centos-mirror
# follow the rsync log...
journalctl -f -u rsync-centos-mirror

Use a systemd timer unit to periodically execute the service above:

cat > /etc/systemd/system/rsync-centos-mirror.timer <<EOF
[Unit]
Description=Periodically Rsync CentOS Mirror
[Timer]
OnStartupSec=300s
OnUnitInactiveSec=2h
[Install]
WantedBy=multi-user.target
EOF
# enable and start the timer unit
systemctl daemon-reload
systemctl enable --now rsync-centos-mirror.timer
# check the date for next activation
systemctl list-timers rsync*

Partial Mirror

# create a list of packages to include
packages=$(
        rpm -qa --qf '\t%{NAME}.%{ARCH}\n' \
                | head -n10 \
                | tr -d '\t' \
                | tr '\n' ' ' \
                | sort \
)
# write a configuration file
cat > reposync.conf <<EOF
[main]
gpgcheck=1
reposdir=/dev/null
[base]
name = base
baseurl = http://mirror.centos.org/centos-7/7/os/x86_64/
enabled = 1
gpgcheck = 1
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
includepkgs=$packages
EOF
mkdir packages
reposync --config=reposync.conf \
         --downloadcomps \
         --newest-only \
         --gpgcheck \
         --arch=x86_64 \
         --repoid=base \
         --download_path=packages/

Custom Repository

Create a local repository to host RPM packages:

>>> yum -y install yum-utils createrepo                       # install the tools
>>> path=/var/www/html/repo                                   # directory holding the repository
>>> mkdir -p $path && createrepo $path                        # intialize the package repository
## move RPM packages into $path
>>> createrepo --update $path                                 # update once packages have been added

Package Automation

Packit

Packit ⁶ help software maintainers to build RPM packages…

…integrates with software repositories hosted on GitHub, GitLab or Pagure ⁷
…uses Copr to build RPMs package from code changes in a software repository
…runs test in the Test Farm ⁸ infrastructure (a scalable Testing System as a Service)
…allows to seamlessly move packages downstream to Fedora Linux & CentOS Stream
…provides the packit CLI ⁹

Copr

Copr ¹⁰ is build system for 3rd party packages…

…short for COPR (Cool Other Package Repositories)
…allows contributors to create RPM packages
…share RPM packages within a public YUM repositories
…automatic management of program version, package release & changelog
…automatic building of new package versions
…keep packages updated by interacting only with a Git repository
Integrates with… tito ¹¹ for managing RPM packages

Search for existing repositories on the public Copr front-end ¹² of the Fedora Project ¹³

OBS

SUSE Open Build Service (OBS) ¹⁴

…public instance ¹⁵ available from the OpenSUSE Project
…documentation in the OBS users guide ¹⁶
Service to build & distribute packages…
- …from source …automatic & reproducible
- …web-interface, API, and osc CLI

References

Packaging for Beginners, Fedora NEST, 2022/10
RPM Packaging Guide, GitHub
System Administrator’s RPM Guide, Fedora Project
Package Maintenance Guide, Fedora Project
Fedora Versioning Guidelines, Fedora Project
How to Easily Patch Fedora Packages, Fedora Project

Footnotes

Introduction to RPM Packaging, OpenSUSE Project
https://documentation.suse.com/sbp/all/html/SBP-RPM-Packaging/index.html ↩︎
Fedora Package Search, Fedora Project
https://packages.fedoraproject.org/↩︎
What is a SPEC File?, RPM Documentation
https://rpm-packaging-guide.github.io/#what-is-a-spec-file ↩︎
Spec file formant, RPM Documentation
https://rpm-software-management.github.io/rpm/manual/spec.html ↩︎
Mock Documentation, GitHub
https://rpm-software-management.github.io/mock ↩︎
Packit Home-Page
https://packit.dev/↩︎
Packit Onboarding Guide
https://packit.dev/docs/guide ↩︎
Testing Farm Documentation
https://docs.testing-farm.io ↩︎
Packit CLI, Packit Documentation
https://packit.dev/docs/cli ↩︎
Copr Documentation, Fedora Project
https://docs.pagure.org/copr.copr ↩︎
Tito Source Code, GitHub
https://github.com/rpm-software-management/tito ↩︎
Copr Web Front-end, Fedora Project
https://copr.fedorainfracloud.org ↩︎
Copr Section, Fedora Infrastructure Documentation
https://docs.fedoraproject.org/en-US/infra/sysadmin_guide/copr/↩︎
Open Build Service
https://openbuildservice.org ↩︎
OpenSUSE Build Service
https://build.opensuse.org ↩︎
OBS User Guide, SUSE
https://openbuildservice.org/help/manuals/obs-user-guide/↩︎

--- title: RPM Packages & Yum Package Repositories categories: - Linux date: 2009/08/12 date-modified: 2024/08/27 toc-expand: 3 --- # Overview The name RPM [^luKXb] refers to the `.rpm` file format and `rpm` program itself: - Binary RPM packages contain compiled software for a specific architecture - Bundles libraries, executables, data, and configuration files required - Includes scripts to deploy and integrate the component... - ...configuration generators, start and stop system services, etc - Source RPM packages`.src.rpm` contain source code to build binary package [^luKXb]: Introduction to RPM Packaging, OpenSUSE Project <https://documentation.suse.com/sbp/all/html/SBP-RPM-Packaging/index.html> Packages use following convention for versioning: ```txt name name.arch name-[epoch:]version name-[epoch:]version-release name-[epoch:]version-release.arch ``` Search for RPM packages [^L4jjD] in the repositories of the Fedora Project. [^L4jjD]: Fedora Package Search, Fedora Project <https://packages.fedoraproject.org/> ## `rpm` Command `rpm` **query** command ...`-q`/`--query` aliased to `rpmquery` ```bash rpm -qi PACKAGE_NAME # general information about a package rpm -ql PACKAGE_NAME # list file in an installed package rpm -qR PACKAGE_NAME # package dependencies and recommendations rpm -qf FILE # query package owning FILE ``` ## Verify ```bash # Verify all installed packages rpm -Va # Verify a specific package, verbose rpm -Vv $package ``` Using RPM to Verify Installed Packages: - Check if system accurately reflects the contents of the RPM database - Don't be too surprised if `rpm -Va` turns up a surprising number of files that failed verification… - …some attributes change during normal system operation - …system configuration different compared to builders environment What Does it Verify? Output is a string of 9 characters, a possible attribute marker Output | Description -|- `S` | Size `M` | Mode …file permissions `5` | MD5 Checksum …check if all files it manipulates `D` | Major Number & Minor Number `L` | Symbolic Link String …is the file's symbolic link contents `U` | Owner `G` | Group `T` | Modification Time `P` | Capabilities differ - Attribute flags will only appear if there is a problem… - …otherwise `.` (period) means the test passed - `missing` file expected by a package Attribute markers (left to the filename) Marker | Description -|- `a` | Build side-effect file (such as buildid links) `c` | Configuration file `d` | Documentation file `g` | Ghost file (i.e. the file contents are not included in the package payload) `l` | License file `m` | File missing, is not a verify failure `r` | Readme file ## `rpmdb` In case procedures fail due to a corrupted `rpmdb` files... ```txt error: rpmdb: BDB0113 Thread/process 1394101/139863873272704 failed: BDB1507 Thread died in Berkeley DB library error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery error: cannot open Packages index using db5 - (-30973) error: cannot open Packages database in /var/lib/rpm Error: Error: rpmdb open failed ``` ...try `rpm --rebuilddb` # RPM Package Build Install development tools ```bash # RPM and build tools... sudo dnf install -y @rpm-development-tools # Fedora tools... sudo dnf install git fedora-packager fedora-review # become member of the mock group sudo usermod -a -G mock $USER ; newgrp mock ``` **DO NOT build packages as root!** Directories `~/rpmbuild/{SPECS,SOURCES,BUILD,RPMS,SRPMS}` - `SPECS` RPM .spec files... - `SOURCES` source code archives of target software - `RPMS` binary RPM packages - `SRPMS` source RPMs (SRPMs) - `BUILD` used during the RPM build process...temporary storage `rpmdev-setuptree` ...set up a SRPM build environment ## RPM Spec ```bash # create a new skeleton spec file rpmdev-newspec ${name}.spec ``` The "Recipe" for creating an RPM package is a `.spec` file [^FQlpp]: [^FQlpp]: What is a SPEC File?, RPM Documentation <https://rpm-packaging-guide.github.io/#what-is-a-spec-file> * Contain the package name, version, RPM revision number - …steps to build, install, and clean a package - …a `changelog` to communicate development details * Multiple packages can be built from a single RPM spec file Contents of a `.spec` file [^ndZK2]: [^ndZK2]: Spec file formant, RPM Documentation <https://rpm-software-management.github.io/rpm/manual/spec.html> * Preamble ...series of **metadata** items * Body contains the main part of the instructions...**sections**... - `%description` full description of the software packaged - `%prep` commands to prepare the software to be built - `%build` commands for actually building the software into machine code - `%check` commands to test the software - `%changelog` record of changes that have happened to the package ### Preamble Example: ```txt Name: prometheus-slurm-textfile Version: 2.0^202404070716 Release: el8 BuildArch: noarch ``` Includes following… - `Name` …no white space …may include `-` - `Version` of the package - …alphanumeric characters 1.1 - …optionally segmented …handle pre- & post-releases - `~` force sorting lower than base …example 1.1~201601 < 1.1 - `^` force sorting higher than base …example 1.1^201601 > 1.1 - `Epoch` …override normal version-release sorting - `Release` …distinguishing between different builds …typically the release version of the distribution - `BuildArch` …architecture of the resulting binary - …`x86_64` typical Intel & AMD processors - …`noarch` …architecture independent ### `%install` Copying the desired build artifacts... - ...from `%builddir` (where the build happens) - ...to `%buildroot` (directory structure with the files to be packaged) ### `%files` List of files that will be installed from the package Macros in the `%files` section... - `%license` ...identifies the file listed as a `LICENSE` file - `%doc` ...file listed as documentation ## `rpmbuild` Build package on the host platform...options... - `-bb` build a binary package - `-bs` build a source package - `-ba` build a binary & source package ```sh # copy input artifacts if required... cp -vr ./* ~/rpmbuild/BUILD # build a package from a RPM spec rpmbuild -ba $package.spec # list all package in the build directory ls ~/rpmbuild/**/**/*.rpm ``` ### Macros Evaluate arbitrary strings containing macros on the command line... ```sh rpm --eval "%{_arch}" # ...provide additional (or overwrite) macros rpm --define "_release el8" --eval "%{_release}" # ...display all build-in macros rpm --showrc ``` Lists macros which are widely used... - `%{_sysconfdir}` ...defaults to `/etc` - `%{_bindir}` ...defaults to `/usr/bin` - `%{_sbindir}` ...defaults to `/usr/sbin` - ...for the build... - `%{_topdir}` ...`%{getenv:HOME}/rpmbuild` - `%{_buildrootdir}` ...`%{_topdir}/BUILDROOT` - `%{buildroot}` ...`%{_buildrootdir}/%{name}-%{version}-%{release}.%{_arch}` - `%{_builddir}` ...`%{_topdir}/BUILD` Override the distribution macros in the `~/.rpmmacros` `%{optflags}` macro contains flags that determine `CFLAGS`, `CXXFLAGS`, `FFLAGS`... ```sh # ...default build flags rpm --eval "%{optflags}" # ...value of the LDFLAGS environment variable set by build systems rpm -E "%{build_ldflags}" ``` Consume a macro within the RPM Spec file... ```sh # ... Version: 1.0 Release: %{_release} BuildArch: noarch # ... ``` Use the command-line **option `--define`** to pass macro with value... ```sh rpmbuild --define='_release el8' #... ``` ### Inspect Inspect package content... ```sh # show package content rpm -qipl $package.rpm # list file in package rpm2cpio $package.rpm | cpio -t # extract a single file rpm2cpio $package.rpm | cpio -idv ./$path/$file ``` ## Mock Mock [^jbFb8] buils RPM packages for different architectures/platforms... - ...creates `chroot`s and builds packages within... - ...“pristine build environment”, decoupled from the host using containers - `mockchain`...build chains of packages that depend on each other [^jbFb8]: Mock Documentation, GitHub <https://rpm-software-management.github.io/mock> Mock target platform configurations... ```sh # for example RockyLinux find /etc/mock -name 'rocky*' # ..or mock --list-chroots ``` Build from a source package ```sh mock -r fedora-36-x86_64 $package.src.rpm ``` ### Manual Customisation Modifying a chroot... ```sh # ...install packaghe into a chroot mock -r $mock_config --install $package_name # or /path/to/rpm # ...--no-clean option is necessary... # ...since by default the chroot is cleaned prior to building mock -r $mock_config --no-clean /path/to/srpm # copy file between host anc chroot mock -r $mock_config --copyin /path/on/host /path/in/chroot mock -r $mock_config --copyout /path/in/chroot /path/on/host ``` Run commands in the chroot ...user `--` separator ```sh # ...start an interactive shell mock -r $mock_config --shell # ...execute a command mock --shell -- ls -l '*' ``` ### Build Options Pass build configuration options to `rpmbuild`... ```sh # ...enable a build option mock ... --with=selinux ... # ...disable a build option mock ... --without=pam ... ``` - Both options above can be used multiple times - Pass arbitral additional options with `--rpmbuild-opts` - Specify an RPM macro file with `--macro-file` ### Debugging Check the `build.log` to determine issues... - ...use option `--no-cleanup-after` to preserve the build environment in the chroot - `--clean` to purge the chroot tree - `--no-clean` Do not clean chroot before building package ## FedPkg Clone the Fedora package configuration: ```sh # checkout the Slurm packages without identification fedpkg clone --anonymous $package # build the package fedpkg mockbuild # list the newly build packages (latest Fedora...Rawhide) find results_$package/ -name '*.rpm' ``` Using a specific branch... ```sh # ...switching to the EPEL 8 branch beforehand fedpkg switch-branch epel8 # List file with changes since epel8 git diff --name-only epel8..rawhide # List changes since epel8 git diff epel8..rawhide ``` Target a specific platform... ```sh fedpkg mockbuild --root rocky+epel-8-x86_64 ``` # Local Repositories A repository is: - A collection of packages. - A set of metadata files (the information present in the RPM file). Allows the package manager to operate with the repository without having stored all packages locally. - RPM package files are retrieved on demand at installation time. ## Local Upstream Mirror Simple **HTTP server** setup: ```bash yum -y install httpd rm /etc/httpd/conf.d/welcome.conf systemctl enable --now httpd # grant access to the HTTP port, or disable the firewall firewall-cmd --permanent --add-service=http && firewall-cmd --reload systemctl stop firewalld && systemctl disable firewalld # disable SELinux >>> grep ^SELINUX= /etc/selinux/config SELINUX=disabled >>> setenforce 0 && sestatus ``` Utilities to install: * `reposync` - used to synchronize remote yum repositories to a local directory, using yum to retrieve the packages * `createrepo` - Create repomd (xml-rpm-metadata) repository ```bash dnf -y install yum-utils createrepo ``` Synchronize CentOS YUM repositories to the local directories ```bash mkdir -p /var/www/html/centos/{base,updates} reposync --downloadcomps \ --plugins \ --gpgcheck \ --download-metadata \ --repoid=base \ --repoid=updates \ --download_path=/var/www/html/centos # create a new repodata for the local repositories for repo in base updates do createrepo --verbose\ --update \ --groupfile comps.xml \ /var/www/html/centos/$repo done ``` Create a systemd service unit [2] to execute `reposync` and `createrep`: ```bash cat > /etc/systemd/system/reposync-centos.service <<EOF [Unit] Description=Mirror package repository [Service] ExecStart=/usr/bin/reposync -gml --download-metadata -r base -r updates -p /var/www/html/centos ExecStartPost=/usr/bin/createrepo -v --update /var/www/html/centos/base -g comps.xml ExecStartPost=/usr/bin/createrepo -v --update /var/www/html/centos/updates Type=oneshot EOF ``` Use a systemd timer unit [3] to periodically execute the service above: ```bash cat > /etc/systemd/system/reposync-centos.timer <<EOF [Unit] Description=Periodically execute package mirror sync [Timer] OnStartupSec=300s OnUnitInactiveSec=2h [Install] WantedBy=multi-user.target EOF ``` ## Full Mirror Sync Full mirror (of a file-system) including ISO images and network install: ```bash mkdir /var/www/html/centos rsync --verbose \ --archive \ --compress \ --delete \ rsync://linuxsoft.cern.ch/centos \ /var/www/html/centos ``` Note: You can use any CentOS mirror with an `rsync://` endpoint. Create a systemd service unit to execute `rsync`: ```bash # write a unit file to execute rsync cat > /etc/systemd/system/rsync-centos-mirror.service <<EOF [Unit] Description=Rsync CentOS Mirror [Service] ExecStartPre=-/usr/bin/mkdir -p /var/www/html/centos ExecStart=/usr/bin/rsync -avz --delete rsync://linuxsoft.cern.ch/centos /var/www/html/centos Type=oneshot EOF # load the configuration systemctl daemon-reload # start rsync systemctl start rsync-centos-mirror # follow the rsync log... journalctl -f -u rsync-centos-mirror ``` Use a systemd timer unit to periodically execute the service above: ```bash cat > /etc/systemd/system/rsync-centos-mirror.timer <<EOF [Unit] Description=Periodically Rsync CentOS Mirror [Timer] OnStartupSec=300s OnUnitInactiveSec=2h [Install] WantedBy=multi-user.target EOF # enable and start the timer unit systemctl daemon-reload systemctl enable --now rsync-centos-mirror.timer # check the date for next activation systemctl list-timers rsync* ``` ## Partial Mirror ```bash # create a list of packages to include packages=$( rpm -qa --qf '\t%{NAME}.%{ARCH}\n' \ | head -n10 \ | tr -d '\t' \ | tr '\n' ' ' \ | sort \ ) # write a configuration file cat > reposync.conf <<EOF [main] gpgcheck=1 reposdir=/dev/null [base] name = base baseurl = http://mirror.centos.org/centos-7/7/os/x86_64/ enabled = 1 gpgcheck = 1 gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 includepkgs=$packages EOF mkdir packages reposync --config=reposync.conf \ --downloadcomps \ --newest-only \ --gpgcheck \ --arch=x86_64 \ --repoid=base \ --download_path=packages/ ``` ## Custom Repository Create a local repository to host RPM packages: ```bash >>> yum -y install yum-utils createrepo # install the tools >>> path=/var/www/html/repo # directory holding the repository >>> mkdir -p $path && createrepo $path # intialize the package repository ## move RPM packages into $path >>> createrepo --update $path # update once packages have been added ``` # Package Automation ## Packit Packit [^10rm6] help software maintainers to build RPM packages... - ...integrates with software repositories hosted on GitHub, GitLab or Pagure [^cxthR] - ...uses Copr to build RPMs package from code changes in a software repository - ...runs test in the Test Farm [^0V7ZL] infrastructure (a scalable Testing System as a Service) - ...allows to seamlessly move packages downstream to Fedora Linux & CentOS Stream - ...provides the `packit` CLI [^n4ZYb] [^10rm6]: Packit Home-Page <https://packit.dev/> [^0V7ZL]:Testing Farm Documentation <https://docs.testing-farm.io> [^cxthR]: Packit Onboarding Guide <https://packit.dev/docs/guide> [^n4ZYb]: Packit CLI, Packit Documentation <https://packit.dev/docs/cli> ## Copr Copr [^zierp] is build system for 3rd party packages... - ...short for COPR (Cool Other Package Repositories) - ...allows contributors to create RPM packages - ...share RPM packages within a public YUM repositories - ...automatic management of program version, package release & changelog - ...automatic building of new package versions - ...keep packages updated by interacting only with a Git repository - Integrates with... `tito` [^UK69K] for managing RPM packages Search for existing repositories on the public Copr front-end [^ptPey] of the Fedora Project [^ADPfg] [^zierp]:Copr Documentation, Fedora Project <https://docs.pagure.org/copr.copr> [^ptPey]: Copr Web Front-end, Fedora Project <https://copr.fedorainfracloud.org> [^ADPfg]: Copr Section, Fedora Infrastructure Documentation <https://docs.fedoraproject.org/en-US/infra/sysadmin_guide/copr/> [^UK69K]: Tito Source Code, GitHub <https://github.com/rpm-software-management/tito> ## OBS SUSE Open Build Service (OBS) [^J6eAf] - ...public instance [^9c0mZ] available from the OpenSUSE Project - ...documentation in the OBS users guide [^LSTkN] - Service to build & distribute packages... - ...from source ...automatic & reproducible - ...web-interface, API, and `osc` CLI [^J6eAf]: Open Build Service <https://openbuildservice.org> [^9c0mZ]: OpenSUSE Build Service <https://build.opensuse.org> [^LSTkN]: OBS User Guide, SUSE <https://openbuildservice.org/help/manuals/obs-user-guide/> # References - [Packaging for Beginners](https://www.youtube.com/watch?v=aLf4b03PHxM), Fedora NEST, 2022/10 - [RPM Packaging Guide](https://rpm-packaging-guide.github.io), GitHub - [System Administrator's RPM Guide](https://docs.fedoraproject.org/en-US/fedora/latest/system-administrators-guide/RPM/), Fedora Project - [Package Maintenance Guide](https://docs.fedoraproject.org/en-US/package-maintainers/Package_Maintenance_Guide), Fedora Project - [Fedora Versioning Guidelines](https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning), Fedora Project - [How to Easily Patch Fedora Packages](https://blog.aloni.org/posts/how-to-easily-patch-fedora-packages), Fedora Project