Difference between revisions of "Fedora Koji Process"

From Dogtag
Jump to: navigation, search
m (Creating new Versions of Dogtag PKI Packages)
(Verify that all Pagure Issues for a Given Milestone are Closed)
Line 4: Line 4:
 
and moved to that <major>.<minor>.<patch> milestone, or remain open in the <major>.<minor>
 
and moved to that <major>.<minor>.<patch> milestone, or remain open in the <major>.<minor>
 
backlog.
 
backlog.
 +
 +
=== Edit the ~/.rpmmacros file in order to utilize the appropriate %{dist} value: ===
 +
<pre>
 +
# vi ~/.rpmmacros
 +
 +
%dist .fc22
 +
</pre>
  
 
=== Checkout and Build from 'master' PKI Source Branch ===
 
=== Checkout and Build from 'master' PKI Source Branch ===

Revision as of 22:57, 7 March 2018

Creating Dogtag PKI Packages on Fedora Koji

Verify that all Pagure Issues for a Given Milestone are Closed

Verify that all of Pagure issues for a <major>.<minor>.<patch> milestone have been closed and moved to that <major>.<minor>.<patch> milestone, or remain open in the <major>.<minor> backlog.

Edit the ~/.rpmmacros file in order to utilize the appropriate %{dist} value:

# vi ~/.rpmmacros

%dist .fc22

Checkout and Build from 'master' PKI Source Branch

    # dnf update
    
    # git clone git@github.com:dogtagpki/pki.git
    
    # script -c "USE_TIMESTAMP=0 USE_GIT_COMMIT_ID=0 pki/scripts/compose_dogtag_pki_meta_packages rpms" typescript.meta
    # mv packages packages.meta
    
    # script -c "USE_TIMESTAMP=0 USE_GIT_COMMIT_ID=0 pki/scripts/compose_dogtag_pki_theme_packages rpms" typescript.theme
    # cd packages/RPMS/noarch
    # sudo dnf install ./*.rpm
    # cd ../../..
    # mv packages packages.theme
    
    # script -c "USE_TIMESTAMP=0 USE_GIT_COMMIT_ID=0 pki/scripts/compose_pki_core_packages rpms" typescript.core
    # cd packages/RPMS
    # mkdir combined
    # cp -p */*.rpm combined
    # cd combined
    # rm *debug*.rpm pki-javadoc*.rpm
    # sudo dnf install ./*.rpm
    # cd ../../..
    # mv packages packages.core
    
    # script -c "USE_TIMESTAMP=0 USE_GIT_COMMIT_ID=0 pki/scripts/compose_pki_console_packages rpms" typescript.console
    # cd packages/RPMS/noarch
    # sudo dnf install ./*.rpm
    # cd ../../..
    # mv packages packages.console
    

Execute PKI Smoke Tests (Optional)

    • Optionally install a 389 DS Server.
    • Optionally pkispawn a CA, KRA, OCSP, TKS, and TPS and sanity test them.
    • Optionally test out PKI CLI.
    • Optionally test out PKI Console for CA, KRA, OCSP, and TKS.

Checkout fresh Koji branches

    # mkdir koji
    # cd koji
    # fedpkg clone -B dogtag-pki
    # fedpkg clone -B dogtag-pki-theme
    # fedpkg clone -B pki-core
    # fedpkg clone -B pki-console
    

Setup Local 'rpmbuild' Creation Repos

Download the 'compose' script

The following local 'rpmbuild' repos utilize the 'compose' script:

    #!/bin/bash
    #
    # Authors:
    #     Matthew Harmsen <mharmsen@redhat.com>
    #
    # --- BEGIN COPYRIGHT BLOCK ---
    # This program is free software; you can redistribute it and/or modify
    # it under the terms of the GNU General Public License as published by
    # the Free Software Foundation; version 2 of the License.
    #
    # This program is distributed in the hope that it will be useful,
    # but WITHOUT ANY WARRANTY; without even the implied warranty of
    # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    # GNU General Public License for more details.
    #
    # You should have received a copy of the GNU General Public License along
    # with this program; if not, write to the Free Software Foundation, Inc.,
    # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
    #
    # Copyright (C) 2016 Red Hat, Inc.
    # All rights reserved.
    # --- END COPYRIGHT BLOCK ---
    #
    
    SCRIPT=$0
    
    ######################
    ## Define Functions ##
    ######################
    
    usage() {
        echo
        echo "Usage:  $0 <name> <action> [-a arch] [-d dist] [-h]"
        echo
        echo "        where <action> must be one of the following:"
        echo
        echo "            dry-run  Print contents of variables"
        echo
        echo "            patch    Apply patches to the source tarball"
        echo
        echo "            srpm     Create a patched SRPM from source tarball + patches"
        echo
        echo "            rpms     Create patched RPMS from source tarball + patches"
        echo
        echo "            build    Create patched SRPM and RPMS from source tarball + patches"
        echo "                     (save build tree)"
        echo
        echo "            package  Create patched SRPM and RPMS from source tarball + patches"
        echo "                     (remove build tree)"
        echo
    }
    
    initialize() {
        rm -rf BUILD BUILDROOT RPMS SRPMS
    
        mkdir BUILD RPMS SRPMS
    }
    
    verify_prerequisite() {
        executable=$1
        rv=0
    
        command -v ${executable} >/dev/null 2>&1
        if [ $? -eq 1 ] ; then
            echo
            echo "The bash script \"${SCRIPT}\" requires \"${executable}\";"
            echo "please install the package which provides \"${executable}\"."
            echo
            rv=1
        fi
    
        return ${rv}
    }
    
    ###################################
    ## Verify Required Prerequisites ##
    ###################################
    
    PREREQUISITES=(date lsb_release pwd rpmbuild script)
    
    missing_prerequisites=0
    for prerequisite in "${PREREQUISITES[@]}" ; do
        verify_prerequisite ${prerequisite}
        missing_prerequisites=`expr ${missing_prerequisites} + $?`
    done
    
    if [ ${missing_prerequisites} -ne 0 ] ; then
        usage
        exit 255
    fi
    
    ######################
    ## Define Constants ##
    ######################
    
    ACTIONS=(dry-run patch srpm rpms build package)
    
    ARCHES=(aarch64 armv7hl i686 ppc ppc64 ppc64le s390 s390x x86_64)
    
    DISTRIBUTION=`lsb_release -is`
    
    OS_RELEASE=`lsb_release -rs`
    
    PWD=`pwd`
    
    TIMESTAMP=`date +%Y%m%d%H%M%S`
    
    #################################
    ## Test for Required Arguments ##
    #################################
    
    if [ $# -lt 1 ] ; then
        echo
        echo "Must specify a 'name'!"
        usage
        exit 255
    elif [ $# -lt 2 ] ; then
        echo
        echo "Must specify an 'action'!"
        usage
        exit 255
    fi
    
    ## Test for expected directory infrastructure
    missing_directories=0
    if [ ! -d "${PWD}/SOURCES/" ] ; then
        echo
        echo "The '${PWD}/SOURCES/' directory is missing!"
        echo
        missing_directories=`expr ${missing_directories} + 1`
    fi
    
    if [ ! -d "${PWD}/SPECS/" ] ; then
        echo
        echo "The '${PWD}/SPECS/' directory is missing!"
        echo
        missing_directories=`expr ${missing_directories} + 1`
    fi
    
    if [ ${missing_directories} -ne 0 ] ; then
        usage
        exit 255
    fi
    
    NAME=$1
    
    SPEC=${NAME}.spec
    
    ## Test for valid name
    if [ ! -f "${PWD}/SPECS/${SPEC}" ] ; then
        echo
        echo "The file '${PWD}/SPECS/${SPEC}' is missing!"
        usage
        exit 255
    fi
    
    ACTION=$2
    
    ## Test for valid action
    if ! [[ " ${ACTIONS[@]} " =~ " ${ACTION} " ]]; then
        echo
        echo "Invalid action '${ACTION}'!"
        usage
        exit 255
    fi
    
    TYPESCRIPT=${NAME}-${ACTION}.${TIMESTAMP}
    
    ##################
    ## Set Defaults ##
    ##################
    
    ## Defaults to the architecture of your current machine
    ARCH=`uname -m`
    
    ## Defaults to the distribution of your current machine
    ##
    ##     NOTE:  Double quotation marks '"' must surround values beginning
    ##            with '-' to prevent '=-' misinterpretation and to still
    ##            allow shell expansion of values which contain variables.
    ##
    if [ "${DISTRIBUTION:0:6}" == "CentOS" ] ; then
        DIST_RELEASE=${OS_RELEASE%.*}
        DIST=.el${DIST_RELEASE}.centos
        DIST_OPTION="-d ${DIST}"
        DIST_RPMBUILD_OPTION="--define \"dist ${DIST}\""
    elif [ "${DISTRIBUTION:0:6}" == "Fedora" ] ; then
        DIST=.fc${OS_RELEASE}
        DIST_OPTION="-d ${DIST}"
        DIST_RPMBUILD_OPTION="--define \"dist ${DIST}\""
    elif [ "${DISTRIBUTION:0:6}" == "RedHat" ] ; then
        DIST_RELEASE=${OS_RELEASE%.*}
        DIST=.el${DIST_RELEASE}
        DIST_OPTION="-d ${DIST}"
        DIST_RPMBUILD_OPTION="--define \"dist ${DIST}\""
    else
        DIST=
        DIST_OPTION=
        DIST_RPMBUILD_OPTION=
    fi
    
    #####################
    ## Parse Arguments ##
    #####################
    
    ## Process any remaining command-line options
    OPTIND=3
    while getopts "a:d:h" option; do
        case "${option}" in
            a)
                arch=${OPTARG}
                ## Test for supported architecture
                if [[ " ${ARCHES[@]} " =~ " ${arch} " ]]; then
                    ARCH=${arch}
                else
                    echo
                    echo "Unsupported architecture '${arch}'!"
                    usage
                    exit 255
                fi
                ;;
            d)
                DIST=${OPTARG}
                DIST_OPTION="-d ${DIST}"
                DIST_RPMBUILD_OPTION="--define \"dist ${DIST}\""
                ;;
            h)
                usage
                exit 0
                ;;
            *)
                usage
                exit 255
                ;;
        esac
    done
    shift $((OPTIND-1))
    
    ##################################
    ## Perform the specified action ##
    ##################################
    
    if [ "${ACTION}" == "dry-run" ] ; then
        ## Print contents of variables
        echo
        echo "COMMAND = '${SCRIPT} ${NAME} ${ACTION} -a ${ARCH} ${DIST_OPTION}'"
        echo
        echo "    ACTION        = '${ACTION}'"
        echo "    ACTIONS       = '(${ACTIONS[@]})'"
        echo "    ARCH          = '${ARCH}'"
        echo "    ARCHES        = '(${ARCHES[@]})'"
        echo "    DIST          = '${DIST}'"
        echo "    DISTRIBUTION  = '${DISTRIBUTION}'"
        echo "    NAME          = '${NAME}'"
        echo "    OS_RELEASE    = '${OS_RELEASE}'"
        echo "    PREREQUISITES = '(${PREREQUISITES[@]})'"
        echo "    SPEC          = '${SPEC}'"
        echo "    TIMESTAMP     = '${TIMESTAMP}'"
        echo "    TYPESCRIPT    = '${TYPESCRIPT}'"
        echo
    elif [ "${ACTION}" == "patch" ] ; then
        ## Apply patches to the source tarball
        initialize
        script -c "rpmbuild --define \"_topdir `pwd`\" ${DIST_RPMBUILD_OPTION} --target ${ARCH} -bp SPECS/${SPEC}" ${TYPESCRIPT}
    elif [ "${ACTION}" == "srpm" ] ; then
        ## Create a patched SRPM from source tarball + patches
        initialize
        script -c "rpmbuild --define \"_topdir `pwd`\" ${DIST_RPMBUILD_OPTION} --target ${ARCH} -bs SPECS/${SPEC}" ${TYPESCRIPT}
    elif [ "${ACTION}" == "rpms" ] ; then
        ## Create patched RPMS from source tarball + patches
        initialize
        script -c "rpmbuild --define \"_topdir `pwd`\" ${DIST_RPMBUILD_OPTION} --target ${ARCH} -bb SPECS/${SPEC}" ${TYPESCRIPT}
    elif [ "${ACTION}" == "build" ] ; then
        ## Create patched SRPM and RPMS from source tarball + patches
        ## (save build tree)
        initialize
        script -c "rpmbuild --define \"_topdir `pwd`\" ${DIST_RPMBUILD_OPTION} --target ${ARCH} -ba SPECS/${SPEC}" ${TYPESCRIPT}
    elif [ "${ACTION}" == "package" ] ; then
        ## Create patched SRPM and RPMS from source tarball + patches
        ## (remove build tree)
        initialize
        script -c "rpmbuild --define \"_topdir `pwd`\" ${DIST_RPMBUILD_OPTION} --target ${ARCH} -ba --clean SPECS/${SPEC}" ${TYPESCRIPT}
    fi
    

dogtag-pki

    # mkdir -p DOGTAG-PKI/archives
    # mkdir -p DOGTAG-PKI/SOURCES
    # mkdir -p DOGTAG-PKI/SPECS
    # cd DOGTAG-PKI/SOURCES
    # wget <latest successful Fedora 28 Koji build of the dogtag-pki SRPM>
    # rpm2cpio ./dogtag-pki*.rpm | cpio -idumv
    # mv ./dogtag-pki.spec ..
    # mv ./dogtag-pki*.rpm ../archives
    # cp -p <path to source checkout>/packages.meta/SRPMS/dogtag-pki*.rpm .
    # rpm2cpio ./dogtag-pki*.rpm | cpio -idumv
    # mv ./dogtag-pki.spec ../SPECS
    # rm ./dogtag-pki*.rpm
    # cd ..
    # diff dogtag-pki.spec <path to Koji>/koji/dogtag-pki/f28/dogtag-pki.spec
      (should be identical)
    
    Edit SPECS/dogtag-pki.spec making any changes necessary
    
    # diff dogtag-pki.spec SPECS/dogtag-pki.spec
    
    # compose dogtag-pki package -d .fc28
    

dogtag-pki-theme

    # mkdir -p DOGTAG-PKI-THEME/archives
    # mkdir -p DOGTAG-PKI-THEME/SOURCES
    # mkdir -p DOGTAG-PKI-THEME/SPECS
    # cd DOGTAG-PKI-THEME/SOURCES
    # wget <latest successful Fedora 28 Koji build of the dogtag-pki-theme SRPM>
    # rpm2cpio ./dogtag-pki-theme*.rpm | cpio -idumv
    # mv ./dogtag-pki-theme.spec ..
    # rm ./dogtag-pki-theme*.tar.gz
    # mv ./dogtag-pki-theme*.rpm ../archives
    # cp -p <path to source checkout>/packages.theme/SRPMS/dogtag-pki-theme*.rpm .
    # rpm2cpio ./dogtag-pki-theme*.rpm | cpio -idumv
    # mv ./dogtag-pki-theme.spec ../SPECS
    # rm ./dogtag-pki-theme*.rpm
    # cd ..
    # diff dogtag-pki-theme.spec <path to Koji>/koji/dogtag-pki-theme/f28/dogtag-pki-theme.spec
      (should be identical)
    
    Edit SPECS/dogtag-pki-theme.spec making any changes necessary
    (this is generally necessary when patches are utilized).
    
    # diff dogtag-pki-theme.spec SPECS/dogtag-pki-theme.spec
    # compose dogtag-pki-theme patch -d .fc28
    
    Fix any problems as necessary.
    
    Remember to check-in any non-patch changes
    (perhaps commented out patch logic to the
     'dogtag-pki-theme.spec' in the source repo)
    
    # compose dogtag-pki-theme package -d .fc28
    

pki-core

    # mkdir -p PKI-CORE/archives
    # mkdir -p PKI-CORE/SOURCES
    # mkdir -p PKI-CORE/SPECS
    # cd PKI-CORE/SOURCES
    # wget <latest successful Fedora 28 Koji build of the pki-core SRPM>
    # rpm2cpio ./pki-core*.rpm | cpio -idumv
    # mv ./pki-core.spec ..
    # rm ./pki-core*.tar.gz
    # mv ./pki-core*.rpm ../archives
    # cp -p <path to source checkout>/packages.core/SRPMS/pki-core*.rpm .
    # rpm2cpio ./pki-core*.rpm | cpio -idumv
    # mv ./pki-core.spec ../SPECS
    # rm ./pki-core*.rpm
    # cd .8
    # diff pki-core.spec <path to Koji>/koji/pki-core/f28/pki-core.spec
      (should be identical)
    
    Edit SPECS/pki-core.spec making any changes necessary
    (this is generally necessary when patches are utilized).
    
    # diff pki-core.spec SPECS/pki-core.spec
    # compose pki-core patch -d .fc28
    
    Fix any problems as necessary.
    
    Remember to check-in any non-patch changes
    (perhaps commented out patch logic to the
     'pki-core.spec' in the source repo)
    
    # compose pki-core package -d .fc28
    

pki-console

    # mkdir -p PKI-CONSOLE/archives
    # mkdir -p PKI-CONSOLE/SOURCES
    # mkdir -p PKI-CONSOLE/SPECS
    # cd PKI-CONSOLE/SOURCES
    # wget <latest successful Fedora 28 Koji build of the pki-console SRPM>
    # rpm2cpio ./pki-console*.rpm | cpio -idumv
    # mv ./pki-console.spec ..
    # rm ./pki-console*.tar.gz
    # mv ./pki-console*.rpm ../archives
    # cp -p <path to source checkout>/packages.console/SRPMS/pki-console*.rpm .
    # rpm2cpio ./pki-console*.rpm | cpio -idumv
    # mv ./pki-console.spec ../SPECS
    # rm ./pki-console*.rpm
    # cd ..
    # diff pki-console.spec <path to Koji>/koji/pki-console/f28/pki-console.spec
      (should be identical)
    
    Edit SPECS/pki-console.spec making any changes necessary
    (this is generally necessary when patches are utilized).
    
    # diff pki-console.spec SPECS/pki-console.spec
    # compose pki-console patch -d .fc28
    
    Fix any problems as necessary.
    
    Remember to check-in any non-patch changes
    (perhaps commented out patch logic to the
     'pki-console.spec' in the source repo)
    
    # compose pki-console package -d .fc28