FESA3 as RPM Installation

More information on RPM is found in Wikipedia: RPM Packet Manager.

FESA3 Core

To be able to easily distribute and install FESA3 on different machines in different environments the FESA3 core is packed into RPM packets. RPM allows to define dependencies to FESA-components as well as third-party-libraries such as boost, ice, middleware-stuff, etc in a clear and secure manner.

Available RPM Packets for FESA3

(fesa-cfg.spec) - used only for release 2.2.2

fesa-base.spec

fesa-environment-gsi.spec

fesa-core.spec

fesa-core-gsi.spec

fesa-codegen.spec

fesa-codegen-gsi.spec

fesa-migration-tools.spec (since FESA3 3.0.0)

fesa-model.spec

fesa-model-gsi.spec

fesa-fwk.spec

fesa-fwk-gsi.spec

fesa-explorer.spec

fesa-tools.spec (since FESA3 3.1.0)

SVN location for RPM spec files: https://www-acc.gsi.de/svn/fesa/framework/trunk/fesa-tools/installer/specs

- Sources / Files for buidling the RPMs are fetched directly from tagged sources in GSI's SVN, e.g. with the scripts

• fesa-tools/installer/getSources/get_fesa_sources.sh
• fesa-tools/installer/getsources/get_fesa-explorer_tarball.sh
• fesa-tools/installer/getsources/get_ctrlib_sources.sh

Notable is the file definitions.txt which holds all the required constants and version numbers for the package build.

FESA3 Dependencies

FESA3 1.x requires the following external libraries:

• libxml2-devel
• libcurl-devel
• openssl-devel

RPM Directory Structure

RPMs require a fixed directory structure:

Name	Purpose
BUILD	contains the contents of the created packets
BUILDROOT	contains the intermediate directores, files for the RPM build process
RPMS	contains the final RPMs built with option -bb, separated per architecture
SOURCES	contains the required sources, packed as tar.gz in this case
SPECS	contains the spec files
SRPMS	contains the final RPMs built with option -ba including sources, separated per architecture

How to build RPMs for FESA3 7.x

Virtual machine: vmla005, login rpmbuild, root

Build from source now uses Git.

Boost 1.69 is required.

CMW_DIRECTORY_CLIENT_SERVERLIST=cmwdev00a.acc.gsi.de:5021

How to build RPMs for FESA3 4.x

Virtual machine: vmla005, login rpmbuild, root

Proceed with Build Order.

How To Build RPMs for FESA3 1.x / 2.x / 3.x

Virtual machine: vml00v, login rpmbuild

Got to the folder /home/rpmbuild/rpmbuild/SPECS and execute:

rpmbuild -bb --target=x86_64 <name>.spec

Build Order

A script helps to automate the RPM build process: build_fesa_rpms.sh. In the script the build and installation order is defined. Since there are interdependencies of RPM packets some have to be installed as root before the build process can proceed. The build scripts will pause at such points and continue once the user has pressed <enter>. A similar script is used to build the CMW RPMs. The user rpmbuild has certain root rights to install the packages during the build process.

• gsi-ctr-timing-lib.spec
• fesa-base.spec
• fesa-cfg.spec
• fesa-codegen.spec
• fesa-model.spec
  • install RPM-files named above as root
• fesa-codegen-gsi.spec
• fesa-graph.spec
• fesa-model-gsi.spec
• fesa-environment-gsi.spec
• fesa-migration-tools.spec
• fesa-tools.spec
• fesa-core.spec
• fesa-core-gsi.spec
• fesa-fwk.spec
• fesa-explorer.spec
• fesa-fwk-gsi.spec

Building Source RPMs (including the sources)

rpmbuild -ba # creates RPM-files that also contain the zipped source folder. Optimal for archiving.

Building RPM packages for different Architectures

rpmbuild -bb --target=x86_64 # creates RPM files without the source code

How To Install

sudo rpm -Uvh /home/rpmbuild/rpmbuild/RPMS/<CPU>/fesa-base.rpm / rpm -Uvh fesa-gsi*.rpm / rpm -Uvh *.rpm # (in folder containing RPMs)

After the Installation (1.x / 2.x)

• Copy (and configure) configuration files from fesa-environment-gsi to /common/usr/export/fesa/global/etc/fesa/[FESA-FWK-Version]/
• Copy fesa3_environment.conf from fesa-environment-gsi to /common/usr/export/fesa/global/etc/fesa/[FESA-FWK-Version]/
• Copy the messages file messages.cfg from fesa-core to /common/usr/export/fesa/global/etc/fesa/[FESA-FWK-Version]/
• Copy the messages file messagesGSI.cfg from fesa-core-gsi to /common/usr/export/fesa/global/etc/fesa/[FESA-FWK-Version]/

To copy	from	to
*.cfg	/opt/fesa/fesa-environment-gsi/[FESA-FWK-Version]/	/common/usr/export/fesa/global/etc/fesa/[FESA-FWK-Version]/
fesa3_environment.conf	/opt/fesa/fesa-environment-gsi/[FESA-FWK-Version]/	/common/usr/export/fesa/global/etc/fesa/[FESA-FWK-Version]/
messages.cfg	/opt/fesa/fesa-core/[FESA-FWK-Version]/	/common/usr/export/fesa/global/etc/fesa/[FESA-FWK-Version]/
messagesGSI.cfg	/opt/fesa/fesa-core-gsi/[FESA-FWK-Version]/	/common/usr/export/fesa/global/etc/fesa/[FESA-FWK-Version]/

Upload the documentaton to the webserver (https://www-acc.gsi.de/dav/documentation/fesa/[FESA-FWK-Version], https://www-acc.gsi.de/dav/documentation/fesa-gsi/[FESA-FWK-Version], https://www-acc.gsi.de/dav/documentation/fesa-gsi/metamodel/[FESA-FWK-Version]).

A basic script (fesa-environment-gsi/specs/post_install.sh, see help for required parameters) performs the required operations.

To adapt

• location of CMW directory server in fesa3_environment.conf, at GSI this is at the moment the following:

export CMW_NAMES=fcmw00a.acc.gsi.de:5021
export CMW_DIR_SERVER=dirSD
export CMW_DIRECTORY_LIST=fcmw00a.acc.gsi.de:5021

To consider

• fesa-version attribute must be the same in all schema files and xml templates (see fesa-model/fesa-model-gsi)
• enumeration values for FESA versions in fesa-model-gsi/src/xml/utility/shared-gsi.xsd must be existing (see XML element LabFesaVersionType), DEV version has to be removed
  
• paths to general makefiles in the fesa-core / fesa-core-gsi makefiles
• FESA Version in GSITemplate.xml

What is installed?

To list the installed FESA packages the following commands help:

rpm -qa | grep fesa
rpm --queryformat '%-30{NAME} %-8{RELEASE} %30{BUILDTIME:date} %{SHA1HEADER}\n' -qa fesa\*$VERSION\*|sort

Important

In 2015 it was decided to remove the timing configuration file for White Rabbit based timing from a FESA release ( Bug 700 ). This file is used for offerering values in an instantiation file and for validation of the instantiation file. For a fully working and usable FESA installation this file needs to be installed in the proper location in a different way apart from the RPM installation process.

Since 2015 the timing groups SAFTlib is used in FESA. Snapshots of this library and its dependencies are done by CSCOFE/FESA in /common/usr/fesa . Still there is no installation package for this.

As of Release 7.0, the SAFTLib / Timing official release libraries in /common/export/timing-rte are used.

How To Uninstall

FESA Libraries and 3rd-Party Dependencies

as root:

• rpm -e --nodeps fesa-gsi (name of rpm package)

• rpm -e $(rpm -qa | grep fesa | cut -d" " -f 1)

• rpm -e $(rpm -qa | grep cern | cut -d" " -f 1)

A script for uninstalling the FESA RPM packets is available in fesa-tools/installer/specs .

FESA Eclipse Plugin

• Use Eclipse uninstall mechanism. This uninstalls the feature only, but doesn't delete the plug-in's jar-files. They have to be removed manually.

Short FESA RPM Installation Manual

Prerequisites

• every FESA developer must be a member of the Linux user group fesa!
• for deployment the following folders must be provided with write permission for all members of the fesa user group
  • /common/fesa/data/development/fesa-data/
  • /common/fesa/data/operational/fesa-data/

  • [root@tst010 ~]# chmod -R g+w /common/fesa/data/
    [root@tst010 ~]# chmod -R u+w /common/fesa/data/
    [root@tst010 ~]# chmod -R u+r /common/fesa/data/

FESA Libraries and 3rd-Party Dependencies

1. Unzip file containing RPM-files into a directory
2. cd2 that directory
3. Install as root: rpm -Uvh fesa-gsi-<version>.rpm

FESA Eclipse Plug-in Installation from Local Directory

1. Use Eclipse Luna as a base (http://www.eclipse.org/downloads/)
2. Unzip file containing the plug-in
3. Start Eclipse Kepler
4. In the Menu select Help>Install New Software...
5. Select button Add... to enter the path to the local folder containing the unzipped FESA plug-in
6. Checkmark the provided FESA plug-in version and click through the installation wizard.
7. Sometimes the installation doesn't work smoothly due to Eclipse's trust checks. In that case the contents of the plugins directory in the unzipped needs to be copied manually into Eclipse's installation folder and the plug-in installation process needs to be repeated.
8. Restart Eclipse after successful installation, add an SVN-Repository location and edit the FESA preferences in Window>Preferences/FESA.

FESA3 Eclipse Plug-In

The FESA3 Eclipse Plug-In is not packed into an RPM packet. It is possible to install the FESA3 Plug-In as described on https://www-acc.gsi.de/wiki/FESA/FESA3Install201 or performing a local install with a matching plug-in version. See also this wiki page for reference: FESA3InstallationClassDevelopers.

Installation Structure

FESA 2.x.x

For the next FESA release (2.x) the following structure will be realized

Paths	Content
/opt/cern/buildsystem/generic/<v>	holds the common buildsystem files for both CMW and FESA, such as Make.common, Make.generic, version number from cern
/opt/cern/buildsystem/fesa/<v>	holds the buildsystem files for FESA3 2.x, such as Make.fesa, Make.parent, Make.targets, Make.class, Make.deployunit
/opt/cern/buildsystem/cmw/<v>	holds the buildsystem files for CMW, such as Make.cmw, Make.parent, Make.targets
/opt/cern/cmw/	holds the CMW libraries such as cmw-directory-client, cmw-log, cmw-rbac, cmw-rda 2.13.2, cmw-rda-idl, cmw-serializer, cmw-util, icethreads (libraries that contain no GSI-specific adaptions)
/opt/fesa/	holds the contents of the FESA RPM packets, such as fesa-codegen(-gsi), fesa-core(-gsi), fesa-model(-gsi), fesa-environment-gsi, fesa-fwk, fesa-explorer, fesa-3rdparty
/opt/gsi/3rdparty/boost/1.54.0	holds 3rd party libraries from boost 1.54, customized for GSI
/opt/gsi/3rdparty/timing/<version>	holds 3rd party libraries for the CERN timing, customized for GSI by Harald
/opt/gsi/3rdparty/zeromq/4.0.4	base middleware for the upcoming FESA release 2.x containing next cmw version
/opt/gsi/3rdparty/gtest/<version>	for a future version of CMW and FESA 2.x.x: common, GSI build of googletest framework

FESA 1.3.1 - deprecated

In order to set up a sustainable installation of the FESA3 framework as well as a coherent CMW installation the following installation structure for FESA3 1.3.1 has been discussed:

Paths	Content
/opt/cern/buildsystem/1.0.3	holds the buildsystem files for FESA3 1.3.1
/opt/cern/buildsystem-cmw/1.0.2/	holds the buildsystem files for CMW, the plan is to use a common buildsystem/opt/cern/cmw for the next release
/opt/cern/cmw/	holds the CMW libraries such as cmw-directory-client, cmw-log, cmw-rbac, cmw-rda 2.13.2, cmw-rda-idl, cmw-serializer, cmw-util, icethreads (libraries that contain no GSI-specific adaptions)
/opt/fesa/	holds the contents of the FESA RPM packets, such as fesa-codegen(-gsi), fesa-core(-gsi), fesa-model(-gsi), fesa-environment-gsi, fesa-fwk, fesa-explorer, fesa-3rdparty
/opt/gsi/3rdparty/boost/1.54.0	holds 3rd party libraries from boost 1.54, customized for GSI
/opt/gsi/3rdparty/timing/1.0.1	holds 3rd party libraries for the CERN timing, customized for GSI
/opt/cmw/	historic, contains the CMW libraries before cmw-rda 2.13.2 as well as the cern timing libraries

How to download the required RPM packages of a release

for i in $(repotrack -u fesa-gsi-2.0.1.i686 fesa-gsi-2.0.1.x86_64|grep -v dists); do curl -O $i; done

Manual download within the aslcluster: http://packages.acc.gsi.de/yum/rhel/6/x86_64/ / http://packages.acc.gsi.de/yum/rhel/6/noarch

http://packages.acc.gsi.de/yum/rhel/7/x86_64/ / http://packages.acc.gsi.de/yum/rhel/7/noarch

FESA3 RPM Recipes

How To Build original FESA3 RPM Packets for a new Release

To provide a new FESA3 release using RPM packages the following steps are required:

1. Prerequisites:
   • existing CMW library installation including CERN's generic buildsystem
   • eca/etherbone stuff in /common/usr/timing/
2. Decide which version number to use for the new FESA3 release
3. Commit the source code to a release branch for the new version, e.g. https://git.acc.gsi.de/FESA3-FWK/framework/src/branch/release-7.0.0
4. Enter the location of this branch in fesa-tools\installer\getSources\git_fesa_sources.sh as

   VERSION=7.0.0
   GIT_REMOTE="git.acc.gsi.de/FESA3-FWK"
   GIT_REPO=framework
   GIT_BRANCH=release-$VERSION

5. Copy the adapted script to the virtual machine

   scp fesa-tools\installer\getSources\get_fesa_sources.sh rpmbuild@vmla005:/home/rpmbuild/getSources

6. Adapt the fesa-* spec files for the new release, especially the definitions.txt file which holds the release version numbers as constants for the RPM spec files
7. Adapt the simple build script build_fesa_rpms.sh for the release version

   VERSION=7.0.0

8. Copy the adapted fesa-* spec files, the definitions.txt file and the build script to the virtual machine

   scp fesa-tools\installer\*.spec definitions.txt build*.sh rpmbuild@vmla005:/home/rpmbuild/rpmbuild/SPECS

9. Log on to the virtual machine reserved for building the FESA3 RPM packets

   ssh rpmbuild@vmla005

10. On vml00v:

    cd /home/rpmbuild/getSources/
    ./get_ctrlib_sources.sh -u <Linux user name> # fetch the sources, !ACC login required
    ./git_fesa_sources.sh 

11. The scripts will fetch the sources from the given SVN location, zip the content for RPM building and copy the zipped packages to /home/rpmbuild/rpmbuild/SOURCES . The password will be prompted at the beginning once.
12. On vml00v:

    cd /home/rpmbuild/rpmbuild/SPECS

13. #Build and install the timing library, example for 64-bit build
    rpmbuild -bb --target=x86_64 gsi-ctr-timing-lib.spec
    # as root:
    rpm -Uvh /home/rpmbuild/rpmbuild/RPMS/x86_64/gsi-ctr-lib-1.0.0-1.0.0-gsi01.x86_64.rpm

14. Run the fesa rpm build script

    ./build_fesa_rpms.sh -c x86_64

    . The script will try to build and install the FESA3 RPM packets. On success the console output is similar to

    ...
    `src/py/codegen-gsi/__pycache__' -> `/home/rpmbuild/rpmbuild/BUILDROOT/fesa-codegen-gsi-2.0.1-2.0.1-gsi01.x86_64//opt/fesa/fesa-codegen-gsi/2.0.1/py/codegen-gsi/__pycache__'
    + ln -s ../../../fesa-codegen/2.0.1/py/codegen /home/rpmbuild/rpmbuild/BUILDROOT/fesa-codegen-gsi-2.0.1-2.0.1-gsi01.x86_64//opt/fesa/fesa-codegen-gsi/2.0.1/py/codegen
    + /usr/lib/rpm/find-debuginfo.sh --strict-build-id /home/rpmbuild/rpmbuild/BUILD/fesa-codegen-gsi-2.0.1
    + /usr/lib/rpm/check-buildroot
    + /usr/lib/rpm/redhat/brp-compress
    + /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip
    + /usr/lib/rpm/redhat/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump
    + /usr/lib/rpm/brp-python-bytecompile
    + /usr/lib/rpm/redhat/brp-python-hardlink
    + /usr/lib/rpm/redhat/brp-java-repack-jars
    Processing files: fesa-codegen-gsi-2.0.1-2.0.1-gsi01.x86_64
    Processing files: fesa-codegen-gsi-2.0.1-debuginfo-2.0.1-gsi01.x86_64
    Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/rpmbuild/rpmbuild/BUILDROOT/fesa-codegen-gsi-2.0.1-2.0.1-gsi01.x86_64
    Wrote: /home/rpmbuild/rpmbuild/RPMS/x86_64/fesa-codegen-gsi-2.0.1-2.0.1-gsi01.x86_64.rpm
    Wrote: /home/rpmbuild/rpmbuild/RPMS/x86_64/fesa-codegen-gsi-2.0.1-debuginfo-2.0.1-gsi01.x86_64.rpm
    Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.3jECiE
    + umask 022
    + cd /home/rpmbuild/rpmbuild/BUILD
    + cd fesa-codegen-gsi-2.0.1
    + /bin/rm -rf /home/rpmbuild/rpmbuild/BUILDROOT/fesa-codegen-gsi-2.0.1-2.0.1-gsi01.x86_64
    + exit 0

    On success the newly built RPM packages will be installed (as root) on the virtual machine. This is necessary to proceed with other packages that depend on previously built packages.
15. Compiler and linker errors, missing file errors etc will be displayed during the RPM build process.
16. The newly built RPM packages are copied to

    /home/rpmbuild/rpmbuild/RPMS

    depending on the architecture they are built for. Source RPMs are copied to

    /home/rpmbuild/rpmbuild/SRPMS

    accordingly.
17. Errors in the FESA sources that appear during the build or while testing the installation should be fixed in the development workspace and committed to the release branch. The newly committed sources should be fetched with the get sources script

    cd /home/rpmbuild/getSources </br> ./get_fesa_sources.sh -u <Linux user name> # fetch the sources, !ACC login required

18. RPM Packets can be built and installed individually. Refer to the documentation above.

For a full installation of FESA3 the following scripts help to complete the installation.

1. The configuration files for FESA3 have to be copied to the export folder, the simple script post_install.sh performs these steps.
2. The internal FESA3 documentation should be copied to the webdav server for external access without Eclipse, the simple script fesa-tools/documentation/uploadDoxygenDocumentation.sh helps to perform this step.
3. Test the new FESA3 installation thouroughly using the integration tests in fesa-core-gsi-integration-test.

Provide Patches for installed FESA3 RPM Packets

Although an RPM package builds successful errors may be discovered later once the FESA3 installation is tested and used. This makes patches for already installed RPM packets necessary. This recipe summarizes the required steps.

1. In the RPM spec file that has to be patched increase the release number from gsi01 (default) to

   Release: gsi02

   or higher numbers if this step has to be repeated.
2. In this spec file provide a valid comment in the changelog at the end of the file, e.g.

   %changelog
   * Wed Sep 03 2014 Solveigh Matthies <s.matthies@gsi.de> - 2.0.1-gsi02
   - Bug 588 - Add support for 64-bit SCUs to startscript
   ... keep older changelog entries

3. Copy the adapted RPM spec file to the virtual machine

   scp fesa-tools\installer\*.spec definitions.txt build*.sh rpmbuild@vml00v:/home/rpmbuild/rpmbuild/SPECS

4. Fetch the new sources from the SVN repository

   cd /home/rpmbuild/getSources </br> ./get_fesa_sources.sh -u <Linux user name> # fetch the sources, !ACC login required

5. Build the patched RPM package using the adapted RPM spec file. On success the console output is similar to

...
`src/py/codegen-gsi/__pycache__' -> `/home/rpmbuild/rpmbuild/BUILDROOT/fesa-codegen-gsi-2.0.1-2.0.1-gsi02.x86_64//opt/fesa/fesa-codegen-gsi/2.0.1/py/codegen-gsi/__pycache__'
+ ln -s ../../../fesa-codegen/2.0.1/py/codegen /home/rpmbuild/rpmbuild/BUILDROOT/fesa-codegen-gsi-2.0.1-2.0.1-gsi02.x86_64//opt/fesa/fesa-codegen-gsi/2.0.1/py/codegen
+ /usr/lib/rpm/find-debuginfo.sh --strict-build-id /home/rpmbuild/rpmbuild/BUILD/fesa-codegen-gsi-2.0.1
+ /usr/lib/rpm/check-buildroot
+ /usr/lib/rpm/redhat/brp-compress
+ /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip
+ /usr/lib/rpm/redhat/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump
+ /usr/lib/rpm/brp-python-bytecompile
+ /usr/lib/rpm/redhat/brp-python-hardlink
+ /usr/lib/rpm/redhat/brp-java-repack-jars
Processing files: fesa-codegen-gsi-2.0.1-2.0.1-gsi02.x86_64
Processing files: fesa-codegen-gsi-2.0.1-debuginfo-2.0.1-gsi02.x86_64
Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/rpmbuild/rpmbuild/BUILDROOT/fesa-codegen-gsi-2.0.1-2.0.1-gsi02.x86_64
Wrote: /home/rpmbuild/rpmbuild/RPMS/x86_64/fesa-codegen-gsi-2.0.1-2.0.1-gsi02.x86_64.rpm
Wrote: /home/rpmbuild/rpmbuild/RPMS/x86_64/fesa-codegen-gsi-2.0.1-debuginfo-2.0.1-gsi02.x86_64.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.3jECiE
+ umask 022
+ cd /home/rpmbuild/rpmbuild/BUILD
+ cd fesa-codegen-gsi-2.0.1
+ /bin/rm -rf /home/rpmbuild/rpmbuild/BUILDROOT/fesa-codegen-gsi-2.0.1-2.0.1-gsi02.x86_64
+ exit 0

1. Install the new RPM package as root.
2. Test the patched FESA3 installation.

---

General RPM commands

How to list the content of an RPM package

rpm -qlp <name>.rpm   # list contents of an RPM file
rpm -qip <name>.rpm   # list package information

How to list installed RPM packages

rpm -qa | grep fesa

How to list content of an installed RPM package

rpm -ql <name>, e.g. rpm -ql fesa-fwk-3.1.0-3.1.0-gsi01.x86_64

How to view information of an installed RPM package

rpm -qi <name of installed RPM package>, e.g. rpm -qi fesa-fwk-3.1.0-3.1.0-gsi01.x86_64

---

Historic

FesaInstallationSuggestion