General Release Process

This is a guide how to perform a full release.

Workflow

All steps are explained below!

Workflow Step Explanations

In advance: Release Preparation by developers

1. Ensure that maven has access to
   • Git
     • If you are working on the asl cluster, ensure your kerberos ticket is still valid (so your session is not older than ~10h) use the commands
       • klist to show your current ticket
       • kdestroy to delete your current ticket (if the expiration time is very soon or already in the past)
       • kinit to create a new ticket (requires cluster acoount password)
   • nexus (see also Maven and Eclipse)
2. Synchronise your project with Git (Refresh and execute Team -> Fetch from Upstream)
   • Handle upstream changes by using Team -> Pull and merging them
3. Fix pom
   • Fix parent version
   • Fix infos in pom (compare with the template pom)
   • Check if dependencies are really needed
   • Fix dependency versions (run command mvn versions:display-property-updates versions:display-plugin-updates versions:display-parent-updates versions:display-dependency-updates -DprocessDependencyManagementTransitive=false)
4. Clean code (no clean up and format for CERN artifacts!)
   
   • Ensure you have eclipse configured as described here
   • Right click on your project -> Source -> Clean Up...
     • choose the ACOAPP Profile
   • Right click on your project -> Source -> Format
   • Run command "mvn clean site" and look at the Checkstyle, Spotbugs, PMD and CPD reports -> the fewer reports the better your code (open website target/site/index.html)
5. Run command "mvn clean verify"
   • Fix errors if necessary
   • Fix Javadoc if necessary
6. Test your code
   • Run your application (using the provided start script in target/bin/)
   • Open all UIs
   • Call at least each function you have changed
   • ...
7. Commit and push to Git

For a full stack release by APP

Update common-lsa-utils-lib language files using LSA@AccDbU

Create up-to-date language files in common-lsa-utils-lib (used by e.g. parammod) by running ParamConfigLanguageFileCreation on master against DEV and committing/pushing uthe results.

Update DB Version in lsa-db-scripts and update LSA_TEST_PRO@AccDbU; run lsa-db-scripts on LSA_TEST_PRO@AccDbU

• checkout lsa-db-scripts, branch master
  
  • master_log_pro_test.xml
  • Commit, Push
• Trigger execution: Jenkins: https://builder.acc.gsi.de/jenkins/job/lsa/job/lsa-test-pro/ -> trigger build (or wait until Jenkins runs it automatically)

Release cscoap-reactor artifacts

See AppHowToReleaseProRelease.

Update mkws-pro scripts

Adapt branch name in:

• https://git.acc.gsi.de/acoapp/cscoap-scripts/src/branch/asl75x/bin/mkws-pro#L43
• https://git.acc.gsi.de/acoapp/cscoap-scripts/src/branch/asl75x/bin/mkws-pro-max#L43
• https://git.acc.gsi.de/acoapp/cscoap-scripts/src/branch/asl75x/bin/mkws-pro-med#L43
• https://git.acc.gsi.de/acoapp/cscoap-scripts/src/branch/asl75x/bin/mkws-pro-model#L43

Add release artifacts to Jenkins

• Delete old release jobs in Jenkins (e.g. the full https://builder.acc.gsi.de/jenkins/job/acoapp/job/R17)
• Use Jenkins script from https://git.acc.gsi.de/schaller/scripts
  • Ask Andreas for details.

Decouple patterns

Using BSS Control, decouple all patterns. (Take notes or make a screenshot of the "before" state!)

Make patterns non resident (don't forget this!)

Using Scheduling App, remove all patterns from schedule. (Take notes or make a screenshot of the "before" state!)

Seriously, do not forget this. It can really hurt if you forget and have to fix things using LSA 2-tier etc. afterwards...

Shut down LSA Server

Ask LSA team!

Update Configs

Update common-config pro branch if necessary, commit and push (deploy happens automatically)

Update DB Version in lsa-db-scripts and update LSA@AccDbP; run lsa-db-scripts on LSA@AccDbP

• checkout lsa-db-scripts, branch master
  
  • master_log_pro.xml
  • Commit, Push
• Trigger execution: Ask LSA team!

Run expert-lsa-db-config on LSA@AccDbP

Ask LSA team!

Update common-lsa-utils-lib language files using LSA@AccDbP

Create up-to-date language files in common-lsa-utils-lib (used by e.g. parammod) by running ParamConfigLanguageFileCreation on release branch against PRO and committing/pushing the results.

To do that, look into ParamConfigLanguageFileCreation#main : There are some lines where the property config location is defined. On release branch, uncomment the line where pro config location is defined, and commit+push that change. Then run the class.

If there are diffs in language files, perform a new release of common-lsa-utils-lib and re-release parammodi as well.

Prepare and perform rollout of LSA server and applications

See AppHowToRelease#Rollout_of_Services_and_Applications

Test Apps

• Check that apps can be started from launcher
• Re-schedule the patterns you previously removed from schedule
• Re-couple the patterns you previously decoupled
• Talk to HKR crew and briefly start a pattern (in BSS Control) to check that it starts running as expected (look in e.g. What's Running)
• ... test stuff.

Close/Restart old apps

Using "WhatsApp Admin App", close or restart old applications that are still running - they are probably broken now anyway (at least if they use LSA).

General Procedures

Here are some explanations for general procedures that can also be useful outside of a full release.

Full Reactor Release

See AppHowToReleaseProRelease (or AppHowToReleaseIntRelease).

How to Snapshot Deploy a Single Artifact

A snapshot deploy can be done multiple times per version.

1. Ensure the version in the pom file has the suffix "-SNAPSHOT" (e.g. version number is 1.0.0-SNAPSHOT)
2. Optionally but recommended: go through Preparation steps
3. Run command "mvn clean deploy"

How to Release a Single Artifact

For e.g. bugfix releases, we do not perform a full reactor release, but instead release just a single artifact/project.

1. Go through Preparation steps
2. If you release from command line it's recommended to close eclipse, disable eclipse' autobuild (Project -> Build Automatically) or close the project in eclipse. Otherwise eclipse might clear the project target folder while maven is using the compiled resources and will fail at this point.
3. Run command (please update the versions given in this example to the one of your project)

   mvn release:prepare release:perform -DreleaseVersion=18.0.0 -DdevelopmentVersion=18.0.1-SNAPSHOT

   If you are doing a hotfix release where the major version does not change (leftmost number) you can just run:

   mvn release:prepare release:perform

Recover from errors

• To recover the project after a failed release attempt you have to call mvn release:rollback release:clean. This will also remove the temporary file release.properties that you might see in this situation.

Rollout of Services and Applications

The general workflow is:

CLEAN → COMPILE → VERIFY → INSTALL → DEPLOY/RELEASE → ROLLOUT

The rollout is done by Ansible (currently here: https://git.acc.gsi.de/fcc-commons/acoapp-ansible) where one can add or modify versions of existing applications in inventories//group_vars/applications.yml. Afterwards Ansible can be executed with the make command. This will apply all changes to the specified environment. (See also AppHowToRolloutWithAnsible for a very basic explanation focused on applications.)