Set-Set Value Comparison Monitoring Prototype

In February 2022, Raphael and Anne created a prototype for a monitoring functionality of Set-Set value comparisons between LSA and devices.

The basic idea is that we display the percentage values of correctly/consistently reported Set values over time. After a Vollversorgung/Fully Supply, the percentage for a Pattern should be in the 90s (100 % is unlikely to be reached when a Pattern has DevAcc devices ). If everything is ok, the percentage trend should remain stable, only having small fluctuations when a Pattern is actively being trimmed. Larger decreases of the percentage values usually indicate that some devices have "lost" their LSA Set values - a Vollversorgung/Full Supply might be in ordner. (See README for more detailed information.)

We acknowledge that having to work with "weird" percentage values below 100 % etc. is not ideal. We have a wiki page for documenting findings about what could be improved and how a "proper" Set-Set value comparison should work.

There is no on-call duty for this tool! Neither for the app nor for the "service".

Code

App

https://git.acc.gsi.de/fcc-applications/expert-beam-proseco-monitoring-app-fx

The app is a normal Fx application. It receives its data from the service (if it is running) and does not perform any checks by itself.

Anne is the expert for this code.

Service

https://git.acc.gsi.de/acoapp/svcp-rest-gsi-reactor/src/branch/master/svcp-rest-gsi-v1

As of 2022-02-23, the service code can be found in its own project. SVCP stands for Setting Value Comparison Prototype .

Previously, the code had been located on a branch in the lsa-rest-gsi-reactor project. Development started in lsa-rest-gsi-reactor because we initially thought we could implement it there, but then decided against it for technical reasons (JAPC cannot easily be accessed from a servlet context). As we wanted to avoid putting in too much work for this prototype at the initial stage, we originally still left the code in this project, but used separate branches that contained only the relevant code for the Set-Set value comparison.

Anne and Raphael are the experts for this code.

"Service" for periodical Set-Set Value Comparison

The actual comparison of Set values is performed periodically in a separate, small "service" that publishes the data via REST.

The base url for PRO is currently: http://asl156.acc.gsi.de:49400/set_value_comparisons_prototype/set_value_comparisons/

There is now also a service for INT with the base url: http://vmla004.acc.gsi.de:42817/set_value_comparisons_prototype/set_value_comparisons

The endpoints are:

• /statistics – Simplified Set value comparison results for the last few hours
• /last_result – HTML report of the last performed comparison
• /results – HTML reports for the last few hours
• /check_now – Triggering a Set value comparison as soon as possible
  • The return value describes if a check has been triggered or not. E.g. if a check is already running, no additional check is triggered.

Usage of vmla004

The aforementioned "service" is currently simply a Java process running on vmla004 . This is a VM that belongs to AP and was meant to be used for client-service tests. As it is a test VM, there are no guarantees about whether or not the VM is running/available, is automatically restarted, etc. ! We accept this for now because of the prototype status of this tool.

Stopping the "service"

If you think this service (or the periodical Set-Set value comparisons it performs) causes problems for production, you can stop it by following the procedure below.

Please note that this means that instances of expert-beam-proseco-monitoring-app-fx will not receive data anymore when the service is stopped.

From the cluster, go on vmla004:

ssh vmla004

Become user "bob":

sudo -u bob bash

Check that the service is running (there should be an entry for svcp-rest-gsi for PRO, and svcp-rest-gsi-int for INT):

ps aux | grep java
bob      21816 14.7 14.4 5013656 560404 ?      Sl   10:43   6:19 /bin/java -Xms128m -Xmx1024m -classpath /common/usr/cscoap/htdocs/applications/svcp-rest-gsi/[...] de.gsi.lsa.core.rest.GrizzlyHttpServerMain
bob      22431  0.0  0.0 112816   984 pts/0    S+   11:26   0:00 grep --color=auto java

Stop the process:

• for PRO:

daemon --name=svcp-rest-gsi --pidfiles="/common/usr/cscoap/opt/svcp-rest-gsi/daemon_pids" --stop

• for INT:

daemon --name=svcp-rest-gsi-int --pidfiles="/common/usr/cscoap/opt/svcp-rest-gsi-int/daemon_pids" --stop

Check again that the service is not running anymore:

ps aux | grep java
bob     22431 0.0 0.0 112816  984 pts/0   S+  11:28  0:00 grep --color=auto java

Starting the "service"

If the service has been stopped, it can be started again by doing the following:

From the cluster, go on vmla004:

ssh vmla004

Become user "bob":

sudo -u bob bash

Check that the service is not running (there should be no entry for svcp-rest-gsi for PRO, respectively svcp-rest-gsi-history-test for INT) :

ps aux | grep java
bob     22431 0.0 0.0 112816  984 pts/0   S+  11:28  0:00 grep --color=auto java

Start the process:

• for PRO:

daemon --name=svcp-rest-gsi --respawn --pidfiles="/common/usr/cscoap/opt/svcp-rest-gsi/daemon_pids" --command="/common/usr/cscoap/htdocs/applications/svcp-rest-gsi/current/bin/svcp-rest-gsi.sh --config=pro "

• for INT:

daemon --name=svcp-rest-gsi-int --respawn --pidfiles="/common/usr/cscoap/opt/svcp-rest-gsi-int/daemon_pids" --command="/common/usr/cscoap/htdocs/applications/svcp-rest-gsi/17.0.0-RC-INT-SNAPSHOT/bin/svcp-rest-gsi.sh --config=int "

Check that the service is running (there should be an entry for svcp-rest-gsi for PRO, respectively svcp-rest-gsi-int for INT):

ps aux | grep java
bob     21816 14.7 14.4 5013656 560404 ?     Sl  11:29  0:01 /bin/java -Xms128m -Xmx1024m -classpath /common/usr/cscoap/htdocs/applications/svcp-rest-gsi/[...] de.gsi.lsa.core.rest.GrizzlyHttpServerMain
bob     22431 0.0 0.0 112816  984 pts/0   S+  11:30  0:00 grep --color=auto java

-- AnnekeWalter - 09 Feb 2022