Release procedure ================= Rules of the game ... * Releases are identified by the standard 3-tuple of version numbers, i.e. ``MAJOR.MINOR.PATCH`` * RPMs additionally have a "release" field that is used to distinguish between pre-releases (alpha/beta) and final releases, and to convey changes between 2 copies of an RPMs that only concern the packaging itself * There are two types of releases: pre-releases (i.e. alpha/beta) and final releases * Final releases are identified solely by the ``MAJOR.MINOR.PATCH`` 3-tuple. All tests **must** pass on all of the supported operating systems before a final release is created. * Pre-releases are used to provide preliminary version of the next release, for testing by collaborators, e.g. to provider early RPMs for a new supported OS, or to confirm a bugfix works as intended before creating a final release. * Compatibility between releases (only applies to final releases): * Fundamental rule: Users should not see any adverse effects if the software is upgraded from version X.Y.Z to X.Y.Z+1, and should not have to recompile their C++ code * So, you should: * *Increment the RPM RELEASE* number if the only changes since the last release concern RPM packaging (e.g. certain files forgotten from specfile in last release, or had incorrect permissions) * *Increment the PATCH* version number if there were **no backward-incompatible changes** to the packages' public **ABI** and to the **expected behaviour** since the last release * *Increment the MINOR* version number if significant features were added or bugs fixed, but the expected behaviour of the public API did not change in a backward incompatible way. * *Increment the MAJOR* version number if the public API changed in a backward compatible way. Tags ---- An **annotated** git tag should be created for each release. The following naming convention should be used for the git tags of releases: * Full release ``vX.Y.Z`` * Pre-release ``prerelease/vX.Y.Z/alpha.N`` This naming convention should stop (possibly numerous) pre-release tags from cluttering up the final tags when viewing lists of tags, since they're separated off in a separate 'subdirectory'. Package versioning ------------------ The convention for the version and release fields (i.e. ``my-package-<version>-<release>.<arch>.rpm``) of the IPbus software RPMs are based on the `Fedora package versioning guidelines <https://fedoraproject.org/wiki/Packaging:Versioning>`__ , as well as similar pages. The ``version`` field contains the package's major, minor and patch version numbers in the usual ``X.Y.Z`` format. The ``release`` field for a normal release (i.e. not pre-release) is a integer greater or equal to ``1``, followed by the OS tag (e.g. ``slc6`` or ``centos7``). However, for a pre-release, the ``release`` field is set equal to ``0.series.N.os``, where ``series`` is equal to to ``alpha`` or ``beta``, and ``N`` is an integer, equal to 1 for the first pre-release in that series, incremented by 1 for each subsequent pre-release in the same series For example, for the X.Y.Z release, and the pre-releases that led up to it: +-----------------------------------+-------------+--------------------+ | (pre-)release | ``version`` | ``release`` | +-----------------------------------+-------------+--------------------+ | First alpha pre-release | ``X.Y.Z`` | ``0.alpha.1.os`` | +-----------------------------------+-------------+--------------------+ | ``N`` th beta pre-release | ``X.Y.Z`` | ``0.beta.N.os`` | +-----------------------------------+-------------+--------------------+ | Final X.Y.Z release | ``X.Y.Z`` | ``1`` | +-----------------------------------+-------------+--------------------+ | Final X.Y.Z release, with changes | ``X.Y.Z`` | ``N``, where N > 1 | | that only affect RPM packaging | | | +-----------------------------------+-------------+--------------------+ .. _pre-tag-checklist: Pre-tag checklist ----------------- 1. (**Not required for pre-releases**) Check that all the tickets are closed for this release at https://github.com/ipbus/ipbus-software/milestones 2. (**Not required for pre-releases**) Check that there were no errors in the automated builds and tests for the commit that you're thinking of tagging 3. Update the RPM version and/or release numbers (in ``Makefile``) for all packages that have changed since the previous release, or depend on other packages that have changed since the previous release * N.B: Always update RPM version numbers to the number of the new release, even if this means increasing the RPM major/minor number by more than 1 * Also, when increasing the version numbers of a package, remember to reset the RPM "package release" value (the ``N`` in RPM name ``<package>-X.Y.Z-N.<arch>.rpm``) to 1. 4. Update ``ci/yumgroups-slc6.xml`` and ``ci/yumgroups-centos7.xml`` if there are new packages in the release (or packages that you want to exclude from this release). Procedure for a normal release ------------------------------ **N.B.** In all commands below, make sure that you replace ``X.Y`` or ``X.Y.Z`` with the ``major.minor`` or full version number for the release. 1. Before even considering making the tag: Go through the :ref:`pre-tag checklist<pre-tag-checklist>` 2. **Make a tag** for the new version, and push to GitHub (the commands below assume that GitHub is the remote named ``origin`` in your repo): .. code-block:: sh git tag -a vX.Y.Z -m "Creating vX.Y.Z release tag" <commit hash> git push origin vX.Y.Z 3. **Wait for the automated build** to run on the tag - its current status can be found at https://gitlab.cern.ch/ipbus/ipbus-software/pipelines ; check that none of the build or test jobs fail. * If any of the jobs do fail, then investigate the cause, and solve the problem before making the release public, removing the tag from the repo if necessary. 4. **YUM repositories** a. *For a new major or minor release:* Copy over the YUM repos created by the automated build .. code-block:: sh ssh ipbus@lxplus mkdir /eos/user/i/ipbus/www/sw/release/X.Y for PLATFORM in slc6_x86_64 centos7_x86_64 centos8_x86_64 ; do mkdir -p /eos/user/i/ipbus/www/sw/release/X.Y/repos/$PLATFORM/{base,updates}/RPMS/ cp -r /eos/user/i/ipbus/www/sw/ci/tags/vX.Y.0/latest/repos/$PLATFORM/* /eos/user/i/ipbus/www/sw/release/X.Y/repos/$PLATFORM/base/RPMS/ cd /eos/user/i/ipbus/www/sw/release/X.Y/repos/$PLATFORM/updates/RPMS createrepo -v . done *For a new patch release:* Copy over the new RPMs created by the automated build, and update each YUM repo .. code-block:: sh ssh ipbus@lxplus for PLATFORM in slc6_x86_64 centos7_x86_64 centos8_x86_64 ; do cd /eos/user/i/ipbus/www/sw/release/X.Y/repos cp /eos/user/i/ipbus/www/sw/ci/tags/vX.Y.Z/latest/repos/$PLATFORM/*X.Y.Z*.rpm $PLATFORM/updates/RPMS/ cd /eos/user/i/ipbus/www/sw/release/X.Y/repos/$PLATFORM/updates/RPMS/ createrepo -v . done b. Test if the YUM repositories are working correctly by attempting to install the new RPMs on a different machine (or a docker container), for each release - using the end-user installation instructions. .. code-block:: sh sudo yum clean all // May be needed sudo yum groupupdate ipbus-sw c. Run unit tests (with same commands as in automated test jobs) on each of the test-install machines. 5. Copy the **doxygen API documentation** from the automated build to the release area .. code-block:: sh ssh ipbus@lxplus mkdir -p /eos/user/i/ipbus/www/sw/release/X.Y/api rsync --recursive --delete --verbose /eos/user/i/ipbus/www/sw/ci/tags/vX.Y.Z/latest/api_uhal/html /eos/user/i/ipbus/www/sw/release/X.Y/api 6. **Update the user's guide** to point to the new release, i.e: * *Major/minor release only:* Update doxygen links * *Major/minor release only:* Update URLs in the repo files that are referenced in the YUM install instructions * Update the "check out and compile uhal from tag" instructions at :ref:`ipbus-user-guide:installation-instructions` 7. **Announcements** a. Write the release announcement post for the IPbus website, and update the version number displayed on the homepage b. Announce the new release by sending an email to hn-cms-online@cern.ch. 8. **CMS Puppet snapshot** Update the CACTUS RPM pool and inform `CMS sysadmins <https://its.cern.ch/jira/browse/CMSONS>`_ to take a new snapshot of the repository. Also update the `Puppet snapshots twiki page <https://twiki.cern.ch/twiki/bin/view/CMS/TriggerSupervisorPuppetSnapshots>`_. Do not ask to deploy unless it's been agreed to. **FIXME** The naming convention for the snapshots is "cactusA.A_uhalB.B_tsC.C.C_YYYYMMDD". To update the pool: .. code-block:: sh ssh cactus@cactus-worker-x cd nightly ./updatePool.py X.Y 9. Finished! Procedure for a pre-release --------------------------- .. todo:: To be filled in.