Fedora Koji Process

From Dogtag
Revision as of 22:51, 7 March 2018 by Mharmsen (talk | contribs) (→‎Creating new Versions of Dogtag PKI Packages)

Jump to: navigation, search

Creating new Versions of Dogtag PKI Packages

Verify that all Pagure Issues for a Given Milestone are Closed

Verify that all of Pagure issues for a <major>.<minor>.<patch> milestone have been closed and moved to that <major>.<minor>.<patch> milestone, or remain open in the <major>.<minor> backlog.

Checkout and Build from 'master' PKI Source Branch

    # dnf update

# git clone git@github.com:dogtagpki/pki.git

# script -c "USE_TIMESTAMP=0 USE_GIT_COMMIT_ID=0 pki/scripts/compose_dogtag_pki_meta_packages rpms" typescript.meta
# mv packages packages.meta

# script -c "USE_TIMESTAMP=0 USE_GIT_COMMIT_ID=0 pki/scripts/compose_dogtag_pki_theme_packages rpms" typescript.theme
# cd packages/RPMS/noarch
# sudo dnf install ./*.rpm
# cd ../../..
# mv packages packages.theme

# script -c "USE_TIMESTAMP=0 USE_GIT_COMMIT_ID=0 pki/scripts/compose_pki_core_packages rpms" typescript.core
# cd packages/RPMS
# mkdir combined
# cp -p */*.rpm combined
# cd combined
# rm *debug*.rpm pki-javadoc*.rpm
# sudo dnf install ./*.rpm
# cd ../../..
# mv packages packages.core

# script -c "USE_TIMESTAMP=0 USE_GIT_COMMIT_ID=0 pki/scripts/compose_pki_console_packages rpms" typescript.console
# cd packages/RPMS/noarch
# sudo dnf install ./*.rpm
# cd ../../..
# mv packages packages.console

Execute PKI Smoke Tests (Optional)

    • Optionally install a 389 DS Server.
    • Optionally pkispawn a CA, KRA, OCSP, TKS, and TPS and sanity test them.
    • Optionally test out PKI CLI.
    • Optionally test out PKI Console for CA, KRA, OCSP, and TKS.

Checkout fresh Koji branches

    # mkdir koji
# cd koji
# fedpkg clone -B dogtag-pki
# fedpkg clone -B dogtag-pki-theme
# fedpkg clone -B pki-core
# fedpkg clone -B pki-console

Setup Local 'rpmbuild' Creation Repos

Download the 'compose' script

The following local 'rpmbuild' repos utilize the 'compose' script: 

    #!/bin/bash
#
# Authors:
#     Matthew Harmsen <mharmsen@redhat.com>
#
# --- BEGIN COPYRIGHT BLOCK ---
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; version 2 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
#
# Copyright (C) 2016 Red Hat, Inc.
# All rights reserved.
# --- END COPYRIGHT BLOCK ---
#

SCRIPT=$0

######################
## Define Functions ##
######################

usage() {
    echo
    echo "Usage:  $0 <name> <action> [-a arch] [-d dist] [-h]"
    echo
    echo "        where <action> must be one of the following:"
    echo
    echo "            dry-run  Print contents of variables"
    echo
    echo "            patch    Apply patches to the source tarball"
    echo
    echo "            srpm     Create a patched SRPM from source tarball + patches"
    echo
    echo "            rpms     Create patched RPMS from source tarball + patches"
    echo
    echo "            build    Create patched SRPM and RPMS from source tarball + patches"
    echo "                     (save build tree)"
    echo
    echo "            package  Create patched SRPM and RPMS from source tarball + patches"
    echo "                     (remove build tree)"
    echo
}

initialize() {
    rm -rf BUILD BUILDROOT RPMS SRPMS

    mkdir BUILD RPMS SRPMS
}

verify_prerequisite() {
    executable=$1
    rv=0

    command -v ${executable} >/dev/null 2>&1
    if [ $? -eq 1 ] ; then
        echo
        echo "The bash script \"${SCRIPT}\" requires \"${executable}\";"
        echo "please install the package which provides \"${executable}\"."
        echo
        rv=1
    fi

    return ${rv}
}

###################################
## Verify Required Prerequisites ##
###################################

PREREQUISITES=(date lsb_release pwd rpmbuild script)

missing_prerequisites=0
for prerequisite in "${PREREQUISITES[@]}" ; do
    verify_prerequisite ${prerequisite}
    missing_prerequisites=`expr ${missing_prerequisites} + $?`
done

if [ ${missing_prerequisites} -ne 0 ] ; then
    usage
    exit 255
fi

######################
## Define Constants ##
######################

ACTIONS=(dry-run patch srpm rpms build package)

ARCHES=(aarch64 armv7hl i686 ppc ppc64 ppc64le s390 s390x x86_64)

DISTRIBUTION=`lsb_release -is`

OS_RELEASE=`lsb_release -rs`

PWD=`pwd`

TIMESTAMP=`date +%Y%m%d%H%M%S`

#################################
## Test for Required Arguments ##
#################################

if [ $# -lt 1 ] ; then
    echo
    echo "Must specify a 'name'!"
    usage
    exit 255
elif [ $# -lt 2 ] ; then
    echo
    echo "Must specify an 'action'!"
    usage
    exit 255
fi

## Test for expected directory infrastructure
missing_directories=0
if [ ! -d "${PWD}/SOURCES/" ] ; then
    echo
    echo "The '${PWD}/SOURCES/' directory is missing!"
    echo
    missing_directories=`expr ${missing_directories} + 1`
fi

if [ ! -d "${PWD}/SPECS/" ] ; then
    echo
    echo "The '${PWD}/SPECS/' directory is missing!"
    echo
    missing_directories=`expr ${missing_directories} + 1`
fi

if [ ${missing_directories} -ne 0 ] ; then
    usage
    exit 255
fi

NAME=$1

SPEC=${NAME}.spec

## Test for valid name
if [ ! -f "${PWD}/SPECS/${SPEC}" ] ; then
    echo
    echo "The file '${PWD}/SPECS/${SPEC}' is missing!"
    usage
    exit 255
fi

ACTION=$2

## Test for valid action
if ! [[ " ${ACTIONS[@]} " =~ " ${ACTION} " ]]; then
    echo
    echo "Invalid action '${ACTION}'!"
    usage
    exit 255
fi

TYPESCRIPT=${NAME}-${ACTION}.${TIMESTAMP}

##################
## Set Defaults ##
##################

## Defaults to the architecture of your current machine
ARCH=`uname -m`

## Defaults to the distribution of your current machine
##
##     NOTE:  Double quotation marks '"' must surround values beginning
##            with '-' to prevent '=-' misinterpretation and to still
##            allow shell expansion of values which contain variables.
##
if [ "${DISTRIBUTION:0:6}" == "CentOS" ] ; then
    DIST_RELEASE=${OS_RELEASE%.*}
    DIST=.el${DIST_RELEASE}.centos
    DIST_OPTION="-d ${DIST}"
    DIST_RPMBUILD_OPTION="--define \"dist ${DIST}\""
elif [ "${DISTRIBUTION:0:6}" == "Fedora" ] ; then
    DIST=.fc${OS_RELEASE}
    DIST_OPTION="-d ${DIST}"
    DIST_RPMBUILD_OPTION="--define \"dist ${DIST}\""
elif [ "${DISTRIBUTION:0:6}" == "RedHat" ] ; then
    DIST_RELEASE=${OS_RELEASE%.*}
    DIST=.el${DIST_RELEASE}
    DIST_OPTION="-d ${DIST}"
    DIST_RPMBUILD_OPTION="--define \"dist ${DIST}\""
else
    DIST=
    DIST_OPTION=
    DIST_RPMBUILD_OPTION=
fi

#####################
## Parse Arguments ##
#####################

## Process any remaining command-line options
OPTIND=3
while getopts "a:d:h" option; do
    case "${option}" in
        a)
            arch=${OPTARG}
            ## Test for supported architecture
            if [[ " ${ARCHES[@]} " =~ " ${arch} " ]]; then
                ARCH=${arch}
            else
                echo
                echo "Unsupported architecture '${arch}'!"
                usage
                exit 255
            fi
            ;;
        d)
            DIST=${OPTARG}
            DIST_OPTION="-d ${DIST}"
            DIST_RPMBUILD_OPTION="--define \"dist ${DIST}\""
            ;;
        h)
            usage
            exit 0
            ;;
        *)
            usage
            exit 255
            ;;
    esac
done
shift $((OPTIND-1))

##################################
## Perform the specified action ##
##################################

if [ "${ACTION}" == "dry-run" ] ; then
    ## Print contents of variables
    echo
    echo "COMMAND = '${SCRIPT} ${NAME} ${ACTION} -a ${ARCH} ${DIST_OPTION}'"
    echo
    echo "    ACTION        = '${ACTION}'"
    echo "    ACTIONS       = '(${ACTIONS[@]})'"
    echo "    ARCH          = '${ARCH}'"
    echo "    ARCHES        = '(${ARCHES[@]})'"
    echo "    DIST          = '${DIST}'"
    echo "    DISTRIBUTION  = '${DISTRIBUTION}'"
    echo "    NAME          = '${NAME}'"
    echo "    OS_RELEASE    = '${OS_RELEASE}'"
    echo "    PREREQUISITES = '(${PREREQUISITES[@]})'"
    echo "    SPEC          = '${SPEC}'"
    echo "    TIMESTAMP     = '${TIMESTAMP}'"
    echo "    TYPESCRIPT    = '${TYPESCRIPT}'"
    echo
elif [ "${ACTION}" == "patch" ] ; then
    ## Apply patches to the source tarball
    initialize
    script -c "rpmbuild --define \"_topdir `pwd`\" ${DIST_RPMBUILD_OPTION} --target ${ARCH} -bp SPECS/${SPEC}" ${TYPESCRIPT}
elif [ "${ACTION}" == "srpm" ] ; then
    ## Create a patched SRPM from source tarball + patches
    initialize
    script -c "rpmbuild --define \"_topdir `pwd`\" ${DIST_RPMBUILD_OPTION} --target ${ARCH} -bs SPECS/${SPEC}" ${TYPESCRIPT}
elif [ "${ACTION}" == "rpms" ] ; then
    ## Create patched RPMS from source tarball + patches
    initialize
    script -c "rpmbuild --define \"_topdir `pwd`\" ${DIST_RPMBUILD_OPTION} --target ${ARCH} -bb SPECS/${SPEC}" ${TYPESCRIPT}
elif [ "${ACTION}" == "build" ] ; then
    ## Create patched SRPM and RPMS from source tarball + patches
    ## (save build tree)
    initialize
    script -c "rpmbuild --define \"_topdir `pwd`\" ${DIST_RPMBUILD_OPTION} --target ${ARCH} -ba SPECS/${SPEC}" ${TYPESCRIPT}
elif [ "${ACTION}" == "package" ] ; then
    ## Create patched SRPM and RPMS from source tarball + patches
    ## (remove build tree)
    initialize
    script -c "rpmbuild --define \"_topdir `pwd`\" ${DIST_RPMBUILD_OPTION} --target ${ARCH} -ba --clean SPECS/${SPEC}" ${TYPESCRIPT}
fi

dogtag-pki

    # mkdir -p DOGTAG-PKI/archives
# mkdir -p DOGTAG-PKI/SOURCES
# mkdir -p DOGTAG-PKI/SPECS
# cd DOGTAG-PKI/SOURCES
# wget <latest successful Fedora 28 Koji build of the dogtag-pki SRPM>
# rpm2cpio ./dogtag-pki*.rpm | cpio -idumv
# mv ./dogtag-pki.spec ..
# mv ./dogtag-pki*.rpm ../archives
# cp -p <path to source checkout>/packages.meta/SRPMS/dogtag-pki*.rpm .
# rpm2cpio ./dogtag-pki*.rpm | cpio -idumv
# mv ./dogtag-pki.spec ../SPECS
# rm ./dogtag-pki*.rpm
# cd ..
# diff dogtag-pki.spec <path to Koji>/koji/dogtag-pki/f28/dogtag-pki.spec
  (should be identical)

Edit SPECS/dogtag-pki.spec making any changes necessary

# diff dogtag-pki.spec SPECS/dogtag-pki.spec

# compose dogtag-pki package -d .fc28

dogtag-pki-theme

    # mkdir -p DOGTAG-PKI-THEME/archives
# mkdir -p DOGTAG-PKI-THEME/SOURCES
# mkdir -p DOGTAG-PKI-THEME/SPECS
# cd DOGTAG-PKI-THEME/SOURCES
# wget <latest successful Fedora 28 Koji build of the dogtag-pki-theme SRPM>
# rpm2cpio ./dogtag-pki-theme*.rpm | cpio -idumv
# mv ./dogtag-pki-theme.spec ..
# rm ./dogtag-pki-theme*.tar.gz
# mv ./dogtag-pki-theme*.rpm ../archives
# cp -p <path to source checkout>/packages.theme/SRPMS/dogtag-pki-theme*.rpm .
# rpm2cpio ./dogtag-pki-theme*.rpm | cpio -idumv
# mv ./dogtag-pki-theme.spec ../SPECS
# rm ./dogtag-pki-theme*.rpm
# cd ..
# diff dogtag-pki-theme.spec <path to Koji>/koji/dogtag-pki-theme/f28/dogtag-pki-theme.spec
  (should be identical)

Edit SPECS/dogtag-pki-theme.spec making any changes necessary
(this is generally necessary when patches are utilized).

# diff dogtag-pki-theme.spec SPECS/dogtag-pki-theme.spec
# compose dogtag-pki-theme patch -d .fc28

Fix any problems as necessary.

Remember to check-in any non-patch changes
(perhaps commented out patch logic to the
 'dogtag-pki-theme.spec' in the source repo)

# compose dogtag-pki-theme package -d .fc28

pki-core

    # mkdir -p PKI-CORE/archives
# mkdir -p PKI-CORE/SOURCES
# mkdir -p PKI-CORE/SPECS
# cd PKI-CORE/SOURCES
# wget <latest successful Fedora 28 Koji build of the pki-core SRPM>
# rpm2cpio ./pki-core*.rpm | cpio -idumv
# mv ./pki-core.spec ..
# rm ./pki-core*.tar.gz
# mv ./pki-core*.rpm ../archives
# cp -p <path to source checkout>/packages.core/SRPMS/pki-core*.rpm .
# rpm2cpio ./pki-core*.rpm | cpio -idumv
# mv ./pki-core.spec ../SPECS
# rm ./pki-core*.rpm
# cd .8
# diff pki-core.spec <path to Koji>/koji/pki-core/f28/pki-core.spec
  (should be identical)

Edit SPECS/pki-core.spec making any changes necessary
(this is generally necessary when patches are utilized).

# diff pki-core.spec SPECS/pki-core.spec
# compose pki-core patch -d .fc28

Fix any problems as necessary.

Remember to check-in any non-patch changes
(perhaps commented out patch logic to the
 'pki-core.spec' in the source repo)

# compose pki-core package -d .fc28

pki-console

    # mkdir -p PKI-CONSOLE/archives
# mkdir -p PKI-CONSOLE/SOURCES
# mkdir -p PKI-CONSOLE/SPECS
# cd PKI-CONSOLE/SOURCES
# wget <latest successful Fedora 28 Koji build of the pki-console SRPM>
# rpm2cpio ./pki-console*.rpm | cpio -idumv
# mv ./pki-console.spec ..
# rm ./pki-console*.tar.gz
# mv ./pki-console*.rpm ../archives
# cp -p <path to source checkout>/packages.console/SRPMS/pki-console*.rpm .
# rpm2cpio ./pki-console*.rpm | cpio -idumv
# mv ./pki-console.spec ../SPECS
# rm ./pki-console*.rpm
# cd ..
# diff pki-console.spec <path to Koji>/koji/pki-console/f28/pki-console.spec
  (should be identical)

Edit SPECS/pki-console.spec making any changes necessary
(this is generally necessary when patches are utilized).

# diff pki-console.spec SPECS/pki-console.spec
# compose pki-console patch -d .fc28

Fix any problems as necessary.

Remember to check-in any non-patch changes
(perhaps commented out patch logic to the
 'pki-console.spec' in the source repo)

# compose pki-console package -d .fc28
Retrieved from "http://www.dogtagpki.org/index.php?title=Fedora_Koji_Process&oldid=31304"