Backport Fedora SLURM Packages

HPC

Published

March 2, 2023

Modified

March 13, 2023

The Fedora projects provides RPM packages [^00] for the SLURM workload management system.

List of Fedora releases with slurm* packages…
- SLURM 20.11 for EPEL 7 & 8
- SLURM 21.08 for Fedora 34-37
- SLURM 22.05 for Fedora 36-39 and EPEL 9
…recent versions of SLURM are be build for build for all supported Fedora releases
…major releases not necessary build for previous EPEL versions

Rebuild SLURM Packages

Install the development tools [^01]:

sudo dnf install git fedora-packager fedora-review
sudo usermod -a -G mock $USER ; newgrp mock

Clone the Fedora package configuration [^00]:

# checkout the Slurm packages without identification
fedpkg clone --anonymous slurm

Rebuild the latest packages….

# build the package
fedpkg mockbuild
# list the newly build packages (latest Fedora...Rawhide)
find results_slurm/ -name '*.rpm'

Build official EPEL8 packages (with Slurm 20.11) by…

# ...switching to the EPEL 8 branch beforehand
fedpkg switch-branch epel8
# List file with changes since epel8
git diff --name-only epel8..rawhide
# List changes since epel8
git diff epel8..rawhide

Mock [^02] configuration for RockyLinux…

>>> find /etc/mock -name 'rocky*'  
/etc/mock/templates/rocky-8.tpl
/etc/mock/rocky+epel-8-aarch64.cfg
/etc/mock/rocky+epel-8-x86_64.cfg
/etc/mock/rocky-8-aarch64.cfg
/etc/mock/rocky-8-x86_64.cfg

21.08 on RockyLinux 8

Adjust slurm.spec [^03] to indicate a rebuild of upstream packages… fedpkg diff

diff --git a/slurm.spec b/slurm.spec
index 91d8be8..1a76c4b 100644
--- a/slurm.spec
+++ b/slurm.spec
@@ -16,7 +16,7 @@
 
 Name:           slurm
 Version:        21.08.8
-Release:        3%{?dist}
+Release:        3~local1%{?dist}
 Summary:        Simple Linux Utility for Resource Management
 License:        GPLv2 and BSD
 URL:            https://slurm.schedmd.com/
@@ -748,6 +748,9 @@ fi
 %systemd_postun_with_restart slurmdbd.service
 
 %changelog
+* Mon Jun 13 2022 Victor Penso <v.penso@email.fqdn> - 21.08.8-3~local1
+- Rebuild of upstream release for EL8.6 + EPEL8
+
 * Mon May 30 2022 Jitka Plesnikova <jplesnik@redhat.com> - 21.08.8-3
 - Perl 5.36 rebuild

Build for RockyLinux 8…

fedpkg mockbuild --root rocky+epel-8-x86_64

Eventually the files above could be modified to build for a specific minor release…

mkdir ~/.config/mock
cp /etc/mock/templates/rocky-8.tpl ~/.config/mock/rocky-8.5.tpl
diff /etc/mock/templates/rocky-8.tpl ~/.config/mock/rocky-8.5.tpl

32,33c32,33
< mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=BaseOS-$releasever
< #baseurl=http://dl.rockylinux.org/pub/rocky/$releasever/BaseOS/$basearch/os/
---
> #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=BaseOS-$releasever
> baseurl=http://dl.rockylinux.org/vault/rocky/8.5/BaseOS/$basearch/os/
40,41c40,41
< mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=AppStream-$releasever
< #baseurl=http://dl.rockylinux.org/pub/rocky/$releasever/AppStream/$basearch/os/
---
> #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=AppStream-$releasever
> baseurl=http://dl.rockylinux.org/vault/rocky/8.5/AppStream/$basearch/os/
48,49c48,49
< mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=PowerTools-$releasever
< #baseurl=http://dl.rockylinux.org/pub/rocky/$releasever/PowerTools/$basearch/os/
---
> #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=PowerTools-$releasever
> baseurl=http://dl.rockylinux.org/vault/rocky/8.5/PowerTools/$basearch/os/
56,57c56,57
< mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=extras-$releasever
< #baseurl=http://dl.rockylinux.org/pub/rocky/$releasever/extras/$basearch/os/
---
> #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=extras-$releasever
> baseurl=http://dl.rockylinux.org/vault/rocky/8.5/extras/$basearch/os/

cp /etc/mock/rocky+epel-8-x86_64.cfg ~/.config/mock/rocky8.5+epel-8-x86_64.cfg
diff /etc/mock/rocky+epel-8-x86_64.cfg ~/.config/mock/rocky8.5+epel-8-x86_64.cfg

1,2c1,2
< include('templates/rocky-8.tpl')
< include('templates/epel-8.tpl')
---
> include('/home/vpenso/.config/mock/rocky-8.5.tpl')
> include('/etc/mock/templates/epel-8.tpl')
4,5c4,5
< config_opts['root'] = 'rocky+epel-8-x86_64'
< config_opts['description'] = 'Rocky Linux 8 + EPEL'
---
> config_opts['root'] = 'rocky8.5+epel-8-x86_64'
> config_opts['description'] = 'Rocky Linux 8.5 + EPEL'

Build specifically against RockyLinux 8.5 + EPEL 8

fedpkg mockbuild --root rocky8.5+epel-8-x86_64

22.05 on RockyLinux 8

Download the source…

wget https://download.schedmd.com/slurm/slurm-22.05.0.tar.bz2

…adjust slurm.spec…

diff --git a/slurm.spec b/slurm.spec
index 91d8be8..d04d141 100644
--- a/slurm.spec
+++ b/slurm.spec
@@ -1,5 +1,5 @@
 # Upstream tarballs use an additional release number
-%global ups_rel 2
+%global ups_rel 1
 
 %if "%{ups_rel}" == "1"
 %global name_version %{name}-%{version}
@@ -15,7 +15,7 @@
 %endif
 
 Name:           slurm
-Version:        21.08.8
+Version:        22.05.0
 Release:        3%{?dist}
 Summary:        Simple Linux Utility for Resource Management
 License:        GPLv2 and BSD

For comparison…

SPECs a very different…

>>> diff -y <(grep '%package' slurm-22.05.0/slurm.spec | sort) <(grep '%package' slurm.spec | sort)
%package contribs                                               %package contribs
%package devel                                                  %package devel
%package example-configs                                      | %package doc
%package libpmi                                               | %package gui
                                                              > %package libs
                                                              > %package nss_slurm
%package openlava                                               %package openlava
%package pam_slurm                                              %package pam_slurm
%package perlapi                                                %package perlapi
                                                              > %package rrdtool
%package slurmctld                                              %package slurmctld
%package slurmd                                                 %package slurmd
%package slurmdbd                                               %package slurmdbd
%package slurmrestd                                             %package slurmrestd
%package slurmsmwd                                            <
%package torque                                                 %package torque

The naive approach will fail…

>>> fedpkg prep                                                                                                                                               
...                                                                                                   
+ /usr/bin/cat /tmp/slurm/slurm_without_cray.patch
+ /usr/bin/patch -p1 -s --fuzz=0 --no-backup-if-mismatch -f
1 out of 1 hunk FAILED -- saving rejects to file src/plugins/switch/Makefile.am.rej
error: Bad exit status from /var/tmp/rpm-tmp.xl7Pg4 (%prep)

Modify slurm_without_cray.patch and adjust the breaking patch with:

# extract the source archive to a dedicated path...
>>> tar -xvf slurm-22.05.0.tar.bz2 -C /tmp/
# ...create the base commit for comparison... 
>>> cd /tmp/slurm-22.05.0 ; git init && git add -f . && git commit -m "Base version"
# ...change the file in question...
>>> $EDITOR src/plugins/switch/Makefile.am
# ...get the patch...
>>> git diff  
diff --git a/src/plugins/switch/Makefile.am b/src/plugins/switch/Makefile.am
index 746550a..dc56a01 100644
--- a/src/plugins/switch/Makefile.am
+++ b/src/plugins/switch/Makefile.am
@@ -1,11 +1,3 @@
 # Makefile for switch plugins
 
 SUBDIRS = none
-
-if WITH_SWITCH_CRAY_ARIES
-SUBDIRS += cray_aries
-endif
-
-if WITH_SWITCH_HPE_SLINGSHOT
-SUBDIRS += hpe_slingshot
-endif

Again with slurm_detect_pmix_v4.patch breaking…

>>> fedpkg prep                                                                                                                                               
...
+ /usr/bin/cat /tmp/slurm/slurm_detect_pmix_v4.patch
+ /usr/bin/patch -p1 -s --fuzz=0 --no-backup-if-mismatch -f
1 out of 1 hunk FAILED -- saving rejects to file auxdir/x_ac_pmix.m4.rej
error: Bad exit status from /var/tmp/rpm-tmp.2dmr8t (%prep)

And so forth…

References

[^00] Slurm RPM Packages, Fedora
[^01] RPM Package Maintenance Guide, Fedora
[^02] Mock Documentation, Red Hat
[^03] Fedora Versioning Guidelines, Fedora

--- title: Backport Fedora SLURM Packages categories: - HPC date: 2023/03/02 date-modified: 2023/03/13 --- The Fedora projects provides RPM packages [^00] for the SLURM workload management system. - List of Fedora releases with [`slurm*` packages][04]... - SLURM 20.11 for EPEL 7 & 8 - SLURM 21.08 for Fedora 34-37 - SLURM 22.05 for Fedora 36-39 and EPEL 9 - ...recent versions of SLURM are be build for build for all supported Fedora releases - ...major releases not necessary build for previous EPEL versions ## Rebuild SLURM Packages Install the development tools [^01]: ```sh sudo dnf install git fedora-packager fedora-review sudo usermod -a -G mock $USER ; newgrp mock ``` Clone the Fedora package configuration [^00]: ```sh # checkout the Slurm packages without identification fedpkg clone --anonymous slurm ``` Rebuild the latest packages.... ```sh # build the package fedpkg mockbuild # list the newly build packages (latest Fedora...Rawhide) find results_slurm/ -name '*.rpm' ``` Build official EPEL8 packages (with Slurm 20.11) by... ```sh # ...switching to the EPEL 8 branch beforehand fedpkg switch-branch epel8 # List file with changes since epel8 git diff --name-only epel8..rawhide # List changes since epel8 git diff epel8..rawhide ``` Mock [^02] configuration for RockyLinux... ```sh >>> find /etc/mock -name 'rocky*' /etc/mock/templates/rocky-8.tpl /etc/mock/rocky+epel-8-aarch64.cfg /etc/mock/rocky+epel-8-x86_64.cfg /etc/mock/rocky-8-aarch64.cfg /etc/mock/rocky-8-x86_64.cfg ``` ## 21.08 on RockyLinux 8 Adjust `slurm.spec` [^03] to indicate a rebuild of upstream packages... `fedpkg diff` ```diff diff --git a/slurm.spec b/slurm.spec index 91d8be8..1a76c4b 100644 --- a/slurm.spec +++ b/slurm.spec @@ -16,7 +16,7 @@ Name: slurm Version: 21.08.8 -Release: 3%{?dist} +Release: 3~local1%{?dist} Summary: Simple Linux Utility for Resource Management License: GPLv2 and BSD URL: https://slurm.schedmd.com/ @@ -748,6 +748,9 @@ fi %systemd_postun_with_restart slurmdbd.service %changelog +* Mon Jun 13 2022 Victor Penso <v.penso@email.fqdn> - 21.08.8-3~local1 +- Rebuild of upstream release for EL8.6 + EPEL8 + * Mon May 30 2022 Jitka Plesnikova <jplesnik@redhat.com> - 21.08.8-3 - Perl 5.36 rebuild ``` Build for RockyLinux 8... ``` fedpkg mockbuild --root rocky+epel-8-x86_64 ``` Eventually the files above could be modified to build for a specific minor release... ```sh mkdir ~/.config/mock cp /etc/mock/templates/rocky-8.tpl ~/.config/mock/rocky-8.5.tpl diff /etc/mock/templates/rocky-8.tpl ~/.config/mock/rocky-8.5.tpl ``` ```diff 32,33c32,33 < mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=BaseOS-$releasever < #baseurl=http://dl.rockylinux.org/pub/rocky/$releasever/BaseOS/$basearch/os/ --- > #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=BaseOS-$releasever > baseurl=http://dl.rockylinux.org/vault/rocky/8.5/BaseOS/$basearch/os/ 40,41c40,41 < mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=AppStream-$releasever < #baseurl=http://dl.rockylinux.org/pub/rocky/$releasever/AppStream/$basearch/os/ --- > #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=AppStream-$releasever > baseurl=http://dl.rockylinux.org/vault/rocky/8.5/AppStream/$basearch/os/ 48,49c48,49 < mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=PowerTools-$releasever < #baseurl=http://dl.rockylinux.org/pub/rocky/$releasever/PowerTools/$basearch/os/ --- > #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=PowerTools-$releasever > baseurl=http://dl.rockylinux.org/vault/rocky/8.5/PowerTools/$basearch/os/ 56,57c56,57 < mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=extras-$releasever < #baseurl=http://dl.rockylinux.org/pub/rocky/$releasever/extras/$basearch/os/ --- > #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=extras-$releasever > baseurl=http://dl.rockylinux.org/vault/rocky/8.5/extras/$basearch/os/ ``` ```sh cp /etc/mock/rocky+epel-8-x86_64.cfg ~/.config/mock/rocky8.5+epel-8-x86_64.cfg diff /etc/mock/rocky+epel-8-x86_64.cfg ~/.config/mock/rocky8.5+epel-8-x86_64.cfg ``` ```diff 1,2c1,2 < include('templates/rocky-8.tpl') < include('templates/epel-8.tpl') --- > include('/home/vpenso/.config/mock/rocky-8.5.tpl') > include('/etc/mock/templates/epel-8.tpl') 4,5c4,5 < config_opts['root'] = 'rocky+epel-8-x86_64' < config_opts['description'] = 'Rocky Linux 8 + EPEL' --- > config_opts['root'] = 'rocky8.5+epel-8-x86_64' > config_opts['description'] = 'Rocky Linux 8.5 + EPEL' ``` Build specifically against RockyLinux 8.5 + EPEL 8 ``` fedpkg mockbuild --root rocky8.5+epel-8-x86_64 ``` ## 22.05 on RockyLinux 8 Download the source... ```sh wget https://download.schedmd.com/slurm/slurm-22.05.0.tar.bz2 ``` ...adjust `slurm.spec`... ```diff diff --git a/slurm.spec b/slurm.spec index 91d8be8..d04d141 100644 --- a/slurm.spec +++ b/slurm.spec @@ -1,5 +1,5 @@ # Upstream tarballs use an additional release number -%global ups_rel 2 +%global ups_rel 1 %if "%{ups_rel}" == "1" %global name_version %{name}-%{version} @@ -15,7 +15,7 @@ %endif Name: slurm -Version: 21.08.8 +Version: 22.05.0 Release: 3%{?dist} Summary: Simple Linux Utility for Resource Management License: GPLv2 and BSD ``` For comparison... * ...[SchedMD/slurm-22.05/slurm.spec](https://github.com/SchedMD/slurm/blob/slurm-22.05/slurm.spec) * ...[fedoraproject.org/f36/slurm.spec](https://src.fedoraproject.org/rpms/slurm/blob/f36/f/slurm.spec) SPECs a very different... ```sh >>> diff -y <(grep '%package' slurm-22.05.0/slurm.spec | sort) <(grep '%package' slurm.spec | sort) %package contribs %package contribs %package devel %package devel %package example-configs | %package doc %package libpmi | %package gui > %package libs > %package nss_slurm %package openlava %package openlava %package pam_slurm %package pam_slurm %package perlapi %package perlapi > %package rrdtool %package slurmctld %package slurmctld %package slurmd %package slurmd %package slurmdbd %package slurmdbd %package slurmrestd %package slurmrestd %package slurmsmwd < %package torque %package torque ``` The naive approach will fail... ```sh >>> fedpkg prep ... + /usr/bin/cat /tmp/slurm/slurm_without_cray.patch + /usr/bin/patch -p1 -s --fuzz=0 --no-backup-if-mismatch -f 1 out of 1 hunk FAILED -- saving rejects to file src/plugins/switch/Makefile.am.rej error: Bad exit status from /var/tmp/rpm-tmp.xl7Pg4 (%prep) ``` Modify `slurm_without_cray.patch` and adjust the breaking patch with: ```sh # extract the source archive to a dedicated path... >>> tar -xvf slurm-22.05.0.tar.bz2 -C /tmp/ # ...create the base commit for comparison... >>> cd /tmp/slurm-22.05.0 ; git init && git add -f . && git commit -m "Base version" # ...change the file in question... >>> $EDITOR src/plugins/switch/Makefile.am # ...get the patch... >>> git diff diff --git a/src/plugins/switch/Makefile.am b/src/plugins/switch/Makefile.am index 746550a..dc56a01 100644 --- a/src/plugins/switch/Makefile.am +++ b/src/plugins/switch/Makefile.am @@ -1,11 +1,3 @@ # Makefile for switch plugins SUBDIRS = none - -if WITH_SWITCH_CRAY_ARIES -SUBDIRS += cray_aries -endif - -if WITH_SWITCH_HPE_SLINGSHOT -SUBDIRS += hpe_slingshot -endif ``` Again with `slurm_detect_pmix_v4.patch` breaking... ```sh >>> fedpkg prep ... + /usr/bin/cat /tmp/slurm/slurm_detect_pmix_v4.patch + /usr/bin/patch -p1 -s --fuzz=0 --no-backup-if-mismatch -f 1 out of 1 hunk FAILED -- saving rejects to file auxdir/x_ac_pmix.m4.rej error: Bad exit status from /var/tmp/rpm-tmp.2dmr8t (%prep) ``` And so forth... ## References [^00] [Slurm RPM Packages][00], Fedora [^01] [RPM Package Maintenance Guide][01], Fedora [^02] [Mock Documentation][02], Red Hat [^03] [Fedora Versioning Guidelines][03], Fedora [00]: https://src.fedoraproject.org/rpms/slurm [01]: https://docs.fedoraproject.org/en-US/package-maintainers/Package_Maintenance_Guide [02]: https://rpm-software-management.github.io/mock [03]: https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning [04]: https://bodhi.fedoraproject.org/updates/?packages=slurm