Release procedure
=================

Rules of the game ...

* Releases are identified by the standard 3-tuple of version numbers, i.e. ``MAJOR.MINOR.PATCH``

  * RPMs additionally have a "release" field that is used to distinguish between pre-releases (alpha/beta) and final releases, and to convey changes between 2 copies of an RPMs that only concern the packaging itself

* There are two types of releases: pre-releases (i.e. alpha/beta) and final releases

  * Final releases are identified solely by the ``MAJOR.MINOR.PATCH`` 3-tuple. All tests **must** pass on all of the supported operating systems before a final release is created.

  * Pre-releases are used to provide preliminary version of the next release, for testing by collaborators, e.g. to provider early RPMs for a new supported OS, or to confirm a bugfix works as intended before creating a final release. 

* Compatibility between releases (only applies to final releases): 

  * Fundamental rule: Users should not see any adverse effects if the software is upgraded from version X.Y.Z to X.Y.Z+1, and should not have to recompile their C++ code
 
  * So, you should:

    * *Increment the RPM RELEASE* number if the only changes since the last release concern RPM packaging (e.g. certain files forgotten from specfile in last release, or had incorrect permissions)  
    * *Increment the PATCH* version number if there were **no backward-incompatible changes** to the packages' public **ABI** and to the **expected behaviour** since the last release
    * *Increment the MINOR* version number if significant features were added or bugs fixed, but the expected behaviour of the public API did not change in a backward incompatible way.
    * *Increment the MAJOR* version number if the public API changed in a backward compatible way.



Tags
----

An **annotated** git tag should be created for each release. The following naming convention should be used for the git tags of releases:

* Full release ``vX.Y.Z``
* Pre-release ``prerelease/vX.Y.Z/alpha.N``

This naming convention should stop (possibly numerous) pre-release tags from cluttering up the final tags when viewing lists of tags, since they're separated off in a separate 'subdirectory'. 



Package versioning
------------------

The convention for the version and release fields (i.e. ``my-package-<version>-<release>.<arch>.rpm``) of the IPbus software RPMs are based on the `Fedora package versioning guidelines <https://fedoraproject.org/wiki/Packaging:Versioning>`__ , as well as similar pages.

The ``version`` field contains the package's major, minor and patch version numbers in the usual ``X.Y.Z`` format.

The ``release`` field for a normal release (i.e. not pre-release) is a integer greater or equal to ``1``, followed by the OS tag (e.g. ``slc6`` or ``centos7``). However, for a pre-release, the ``release`` field is set equal to ``0.series.N.os``, where ``series`` is equal to to ``alpha`` or ``beta``, and ``N`` is an integer, equal to 1 for the first pre-release in that series, incremented by 1 for each subsequent pre-release in the same series

For example, for the X.Y.Z release, and the pre-releases that led up to it:

+-----------------------------------+-------------+--------------------+
| (pre-)release                     | ``version`` | ``release``        |
+-----------------------------------+-------------+--------------------+
| First alpha pre-release           | ``X.Y.Z``   | ``0.alpha.1.os``   |
+-----------------------------------+-------------+--------------------+
| ``N`` th beta pre-release         | ``X.Y.Z``   | ``0.beta.N.os``    |
+-----------------------------------+-------------+--------------------+
| Final X.Y.Z release               | ``X.Y.Z``   |      ``1``         |
+-----------------------------------+-------------+--------------------+
| Final X.Y.Z release, with changes | ``X.Y.Z``   | ``N``, where N > 1 |
| that only affect RPM packaging    |             |                    |
+-----------------------------------+-------------+--------------------+



.. _pre-tag-checklist:

Pre-tag checklist
-----------------

1. (**Not required for pre-releases**) Check that all the tickets are closed for this release at https://github.com/ipbus/ipbus-software/milestones
2. (**Not required for pre-releases**) Check that there were no errors in the automated builds and tests for the commit that you're thinking of tagging 
3. Update the RPM version and/or release numbers (in ``Makefile``) for all packages that have changed since the previous release, or depend on other packages that have changed since the previous release

  * N.B: Always update RPM version numbers to the number of the new release, even if this means increasing the RPM major/minor number by more than 1
  * Also, when increasing the version numbers of a package, remember to reset the RPM "package release" value (the ``N`` in RPM name ``<package>-X.Y.Z-N.<arch>.rpm``) to 1.

4. Update ``ci/yumgroups-slc6.xml`` and ``ci/yumgroups-centos7.xml`` if there are new packages in the release (or packages that you want to exclude from this release).


Procedure for a normal release
------------------------------

**N.B.** In all commands below, make sure that you replace ``X.Y`` or ``X.Y.Z`` with the ``major.minor`` or full version number for the release.

1. Before even considering making the tag: Go through the :ref:`pre-tag checklist<pre-tag-checklist>`


2. **Make a tag** for the new version, and push to GitHub (the commands below assume that GitHub is the remote named ``origin`` in your repo):

  .. code-block:: sh

    git tag -a vX.Y.Z -m "Creating vX.Y.Z release tag" <commit hash>
    git push origin vX.Y.Z


3. **Wait for the automated build** to run on the tag - its current status can be found at https://gitlab.cern.ch/ipbus/ipbus-software/pipelines ; check that none of the build or test jobs fail.

  * If any of the jobs do fail, then investigate the cause, and solve the problem before making the release public, removing the tag from the repo if necessary.


4. **YUM repositories**

  a. *For a new major or minor release:* Copy over the YUM repos created by the automated build

    .. code-block:: sh

      ssh ipbus@lxplus
      mkdir /eos/user/i/ipbus/www/sw/release/X.Y
      for PLATFORM in slc6_x86_64 centos7_x86_64 centos8_x86_64 ; do
        mkdir -p /eos/user/i/ipbus/www/sw/release/X.Y/repos/$PLATFORM/{base,updates}/RPMS/
        cp -r /eos/user/i/ipbus/www/sw/ci/tags/vX.Y.0/latest/repos/$PLATFORM/* /eos/user/i/ipbus/www/sw/release/X.Y/repos/$PLATFORM/base/RPMS/
        cd /eos/user/i/ipbus/www/sw/release/X.Y/repos/$PLATFORM/updates/RPMS
        createrepo -v .
      done

    *For a new patch release:* Copy over the new RPMs created by the automated build, and update each YUM repo

    .. code-block:: sh

      ssh ipbus@lxplus
      for PLATFORM in slc6_x86_64 centos7_x86_64 centos8_x86_64 ; do
        cd /eos/user/i/ipbus/www/sw/release/X.Y/repos
        cp /eos/user/i/ipbus/www/sw/ci/tags/vX.Y.Z/latest/repos/$PLATFORM/*X.Y.Z*.rpm $PLATFORM/updates/RPMS/
        cd /eos/user/i/ipbus/www/sw/release/X.Y/repos/$PLATFORM/updates/RPMS/
        createrepo -v .
      done

  b. Test if the YUM repositories are working correctly by attempting to install the new RPMs on a different machine (or a docker container), for each release - using the end-user installation instructions.

    .. code-block:: sh
 
      sudo yum clean all      // May be needed
      sudo yum groupupdate ipbus-sw


  c. Run unit tests (with same commands as in automated test jobs) on each of the test-install machines.


5. Copy the **doxygen API documentation** from the automated build to the release area

  .. code-block:: sh

    ssh ipbus@lxplus
    mkdir -p /eos/user/i/ipbus/www/sw/release/X.Y/api
    rsync --recursive --delete --verbose  /eos/user/i/ipbus/www/sw/ci/tags/vX.Y.Z/latest/api_uhal/html /eos/user/i/ipbus/www/sw/release/X.Y/api

6. **Update the user's guide** to point to the new release, i.e:

  * *Major/minor release only:* Update doxygen links
  * *Major/minor release only:* Update URLs in the repo files that are referenced in the YUM install instructions
  * Update the "check out and compile uhal from tag" instructions at :ref:`ipbus-user-guide:installation-instructions`

7. **Announcements** 

  a. Write the release announcement post for the IPbus website, and update the version number displayed on the homepage

  b. Announce the new release by sending an email to hn-cms-online@cern.ch.  

8. **CMS Puppet snapshot** Update the CACTUS RPM pool and inform `CMS sysadmins <https://its.cern.ch/jira/browse/CMSONS>`_ to take a new snapshot of the repository. Also update the `Puppet snapshots twiki page <https://twiki.cern.ch/twiki/bin/view/CMS/TriggerSupervisorPuppetSnapshots>`_. Do not ask to deploy unless it's been agreed to. **FIXME** The naming convention for the snapshots is "cactusA.A_uhalB.B_tsC.C.C_YYYYMMDD". To update the pool:

  .. code-block:: sh

    ssh cactus@cactus-worker-x
    cd nightly
    ./updatePool.py X.Y 

9. Finished!



Procedure for a pre-release
---------------------------

.. todo::

  To be filled in.