How to make changes to the database

Project Structure

All LSA database related projects reside in the LSA Gitea Organisation https://git.acc.gsi.de/lsa, usually for configuring the database content (like makerules, element mappings etc.) it is not necessary to check out all of them. If you only want to change the configuration you do not need the 'lsa-db-scripts' project, since it is only needed for making structural changes to the database.

These are the existing projects and their structure from 2017 (To see a live version visit: https://git.acc.gsi.de/lsa?sort=alphabetically&q=lsa-db&tab=):

https://git.acc.gsi.de/lsa?sort=alphabetically&q=lsa-db&tab=
|-- lsa-db-scripts  - EXPERT, generic scripts for all databases like structural changes
`-- expert-lsa-db-config - Data (makerule entries, devices, etc.) (branches for the different databases (master: LSA@AccDbU, int: LSA@AccDbZ, pro: LSA@AccDbP) 
    `-- src
        `-- main
            |-- java
            |   `-- de.gsi.lsa.expert.db.config - EXPERT java code for the importers
            `-- resources
                `-- de.gsi.lsa.expert.db.config
                    |-- accelerator - accelerator specific data
                    `-- global - global data

The lsa-db-scripts project uses a tool called Liquibase that keeps track which scripts were already applied on a database schema and which scripts are missing. It then only applies the missing scripts and updates the configuration. The data is updated by executing the expert-lsa-db-condig java application. It provides a command line friendly interface:

mvn compile exec:java --quiet -Dexec.args="-h"

Preparation

Configuring database passwords (maven authentication only needed for lsa-db-scripts)

Since Liquibase needs access to the database and we configure everything through maven, we also manage the database passwords in maven. The INN Group maintains a Wiki page with a how-to configure authentication information in maven: https://www-acc.gsi.de/wiki/IN/Maven#Authentication

Please follow the guide to add an authentication section for the development and integration database servers to your $HOME/.m2/settings.xml:

  <server>
    <id>AccDbU-LSA</id>
    <username>LSA</username>
    <password> <REPLACE_WITH_ENCRYPTED_DB_PASSWORD> </password>
  </server>
  <server>
    <id>AccDbT-LSA</id>
    <username>LSA</username>
    <password> <REPLACE_WITH_ENCRYPTED_DB_PASSWORD> </password>
  </server>

The passwords are provided by the LSA team on request. Note that we will not provide a password for the production database, because in production the scripts will only be executed by the LSA team.

Checking out the projects in Eclipse

For maintaining the configuration data you only need to check out the expert-lsa-db-config project as maven projects from Git: https://git.acc.gsi.de/lsa/expert-lsa-db-config

Making Changes

Changing the Configuration

To change the configuration, navigate in the resource folder to the appropriate file that contains the information.

Usually the CSV files have a header that describe what is expected. For example, makerules.csv looks like this:

"PARTICLE_TRANSFER_NAME","SOURCE_PARAMETER_TYPE_NAME","DEPENDENT_PARAMETER_TYPE_NAME","MAKERULE_NAME"
"DEFAULT","I","I_POLYNOMIAL","DF2BPSMR"
"DEFAULT","FRF","FRF_POLYNOMIAL","DF2BPSMR"
...

Just open the appropriate CSV file with an editor and update the content. If a line starts with '#' it is treated as a comment.

Changes to the production database (LSA@AccDbP)

The configuration data in expert-lsa-db-config (branch: pro) should only be changed during a release or for other reasons demanded by accelerator operations, like missing features or bug fixing. Any changes during operation must be communicated with the LSA team to ensure that they are still compatible with the LSA version currently used in production. Since this project will use the production release importer versions, the file format might be different from branch master if the importers have been changed since the last production release.

Checking the Changes

For expert-lsa-db-config there is no change report mechanism yet. This application always applies the current state of the resources to the database. However, there is a --dry-run option that just logs what the program would do on the command line.

For lsa-db-scripts, to inquire through Liquibase which changes will be applied the maven targets are "clean install liquibase:status".

You can do this from the command line by navigating into the root of lsa-db-scripts and running:

mvn clean compile liquibase:status

Or from Eclipse:

Right Click on the project
-> Run As
-> Maven Build ...
-> Enter in the 'Goals' field "clean compile liquibase:status"
-> Click the 'Run' button

Executing the Changes

First, commit and push your changes to Git!

After that, executing Liquibase and applying your changes is in principle the same as checking for changes, just use different Maven targets:

mvn clean compile liquibase:update

After running this command, the Maven output should contain the message "BUILD SUCCESS":

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 9.036 s
[INFO] Finished at: 2017-08-09T17:10:43+02:00
[INFO] Final Memory: 47M/794M
[INFO] ------------------------------------------------------------------------

Expert Users

A section for database and importer developers will follow here later.