-- UlrichBecker - 18 Jan 2024

The LM32-Firmware SCU-Control and the DAQ- Linux layer

Please read the topic LM32-developments before.

Table of Contents

The LM32 Firmware for time critical SCU controls.

The aim of the LM32 firmware is to accomplish time critical jobs for this the Llnux host is to slow.
The data exchange between the LM32 firmware and the Linux- host (SaftLib and/or FESA-clases) will made at the time via the working memory of the LM32 which is declared as shared memory and the DDR3 RAM (in the future the SRAM of SCU 4)
In this way, more working memory will be available for the LM32 application respectively for future LM32 tasks.
At the time following tasks will accomplished by the LM32 application SCU-control:

• Supplying the ADDAC function generators with new polynomial data when the concerning funktion- generator had signaled this by a MSI (Message Signaled Interrupt). This polynomial data becomes written by the SaftLib in the shared memory section of the LM32 working memory
  which wastes a lot of expensive LM32- working memory.
• Reads the ADDAC-DAQ data from the concerning ADDAC-DAQ-channel and copy them to the DDR3-RAM respectively in the future in the SRAM of the SCU4 if the concerning ADDAC-DAQ channel had signaled this by a MSI (Message Signaled Interrupt).
• Supplying the MIL function generators with new polynomial data when the concerning MIL-funktion- generator respectively the MIL task- RAM had signaled this by a MSI (Message Signaled Interrupt). This polynomial data becomes written by the SAFT-LIB in the shared memory section of the LM32 working memory which wastes a lot of expensive LM32- working memory.
• Reads C- coefficient (set value) of the concerning function generators polynomial back and the corresponding analog value (actual value) from the MIL AD converter as number tupel which is presented as MIL-DAQ data.
• Copies the MIL-DAQ data together with a WR- timestamp in the shared- memory section of the LM32 working memory or (depending of the makefile variable MIL_DAQ_USE_RAM ) in the DDR3- RAM respectively in the future in the SRAM of the SCU 4.
• Additional the new firmware using FreeRTOS (scu3_control_os.bin only) reads every 10 seconds three temperature sensors connected by the one wire bus, evaluated them and puts a alarm-message in the LM32-log system if a critical temperature had reached.

CAUTION: In contrast to the LM32-base firmware which is preconfigured in the FPGA and doesn't support the handling of DAQs, for each function generator which is controlled by the SaftLib the both belonging DAQ channels for set and actual values becomes automatically switched on and of together with the concerning function generator.
For more on this see below.

Automatic uploading and start of the LM32 application in an SCU for testing purposes

The latest versions of the LM32-binary application (scu3_control.bin and the FreeRTOS version scu3_control_os.bin) are in the directory /common/export/nfsinit/global/lm32-test-apps/ . In this directory you can find the following files:

scu3_control.bin	The classical version (in the future deprecated)
scu3_control_os.bin	The new version using FreeRTOS (beta)
scu_test_init.sh	The start-script. Depending on the leading number of the symbolic link which points to this script, the classical version scu3_control.bin becones started if the number is 95 or the FreeRTOS - version scu3_control_os.bin becomes started if the leading number is 96.

IMPORTANT: In order to synchronize the time stamp counter registers of the DAQ macros in the ADDAC slaves, it is necessary to define a so-called ECA rule before starting respectively uploading the LM32 application. This will be accomplished with the following command, which should also be implemented in the starter script before the LM32 firmware is loaded:

saft-scu-ctl tr0 -d -c 0xDADADADABABABABA 0xFFFFFFFFFFFFFFFF 0 0xCAFEAFFE

This command above is also implemented in the starter-script: "scu_test_init.sh".

When a SCU shall run with one of the both LM32-applications then a symbolic link which points to the start-script "scu_test_init.sh" with a leading two digit number has to be made in the following directory of the concerned SCU: /common/export/nfsinit/scuxl4711/
Example:

ln -s /common/export/nfsinit/global/lm32-test-apps/scu_test_init.sh /common/export/nfsinit/scuxl4711/95_start_scu_control

In this example the classical version of the LM32-application will uploaded in the LM32 and invoked.

ln -s /common/export/nfsinit/global/lm32-test-apps/scu_test_init.sh /common/export/nfsinit/scuxl4711/96_start_scu_control

In this example the new FreeRTOS version of the LM32-application will uploaded in the LM32 and invoked.

Further the start-script scu_test_init.sh invokes the lm32-log-daemon lm32-logd and a logfile becomes created: /var/log/lm32.log . Please read this topic as well.

Makefile variables and compiler switches

CAUTION: If you change any variable or cmpilerswitch in the Makefile, then a make rebuild becomes necessary!

• MIL_DAQ_USE_RAM If this variable defined then the raw data of the MIL-DAQs becomes stored in the DDR3-RAM, otherwise the raw data becomes stored in the working memory of LM32. Changes in the linux part libscu_fg_feedback.a are not necessary this will recognized automatically during the probing.

The classical firmware "scu3_control.bin" without FreeRTOS

Currently this firmware scu3_control.bin is still the LM32-firmware which is used in the production environment.

If you works on the ASL then the documentation of the source code, generated by Doxygen can be found in this folder:

/common/usr/cscofe/doc/scu/lm32-firmware/scu3_control/html

...and the following link should work: file:///common/usr/cscofe/doc/scu/lm32-firmware/scu3_control/html/index.html
otherwise it's a dead link.

Building of the for LM32 executable file "scu3_control.bin"

1. Change in directory gsi_scu/prj/scu-control/lm32-non-os_exe/SCU3/.
2. Build binary file by typing: make.
3. If the binary was successful built then it will be in this directory: gsi_scu/prj/scu-control/lm32-non-os_exe/SCU3/deploy_lm32/result/scu3_control.bin.
4. If the variable SCU_URL specified so you can upload the binary-file by typing make load.

The new firmware "scu3_control_os.bin" using FreeRTOS

Please read the topic FreeRTOS especially for the LM32 using in the SCU before.

If you works on the ASL then the documentation of the source code, generated by Doxygen can be found in this folder:

/common/usr/cscofe/doc/scu/lm32-firmware/scu3_control_os/html

...and the following link should work: file:///common/usr/cscofe/doc/scu/lm32-firmware/scu3_control_os/html/index.html
otherwise it's a dead link.

At the time the new LM32 firmware which used the embedded real-time operating system FreeRTOS, is in the beta state yet.
But it seems to work pretty well. This firmware is intended to replace the old one in the future, especially with regard to the future for the SCU 4.

What is the benefit of the firmware version with FreeRTOS?
Primary, this firmware does everything what the old one does.
In addition, the three temperature sensors becomed read out every 10 seconds by the slow single-wire bus (this are blocking functions) and,
if a critical temperature is exceeded, than an alarm message is issued via the LM32 log system. This could happen if for example when a cooling fan is blocked and so on.
Furthermore in the etherbone-console an animated software fan {'|','/','-','\'} is shown as "still alive" signal. If this not desired, so the source code of that can be deactivated
by commenting out the compiler switch DEFINES += CONFIG_STILL_ALIVE_SIGNAL in the makefile.

Otherwise, there will be many more possibilities for additional functions in the future, because the possibilities with the old firmware have been exhausted.

Building of the for LM32 executable file "scu3_control_os.bin"

1. Change in directory gsi_scu/prj/scu-control/lm32-rtos_exe/SCU3/.
2. Build binary file by typing: make.
3. If the binary was successful built then it will be in this directory: gsi_scu/prj/scu-control/lm32-rtos_exe/SCU3/deploy_lm32/result/scu3_control_os.bin.
4. If the variable SCU_URL specified so you can upload the binary-file by typing make load.

Makefile variables and compiler switches of scu3_control_os.bin

• USE_ADDAC_FG_TASK If this makefile variable defined, then all ADDAC- function generators runs in a FreeRTOS - task, else the ADDAC- function generators runs directly in the interrupt-context like the non-os version scu3-control.bin.
  The belonging compiler switch is: CONFIG_USE_ADDAC_FG_TASK which becomes defined automatically.
  At the moment I don't know yet what the more better solution is. The disadvantage of running ADDAG- function generators in a FreeRTOS task is the higher consuming of LM32 working memory, but the advantage is, that jittering and latency time of the interrupt is as small as possible.
• USE_TEMPERATURE_WATCHER If this makefile variable defined, then the FreeRTOS - task which reads and evaluates the temperature- sensors connected by the on wire bus becomes compiled.
  The belonging compiler switch is: CONFIG_USE_TEMPERATURE_WATCHER which becomes defined automatically.
  NOTE: The MSI which had triggered to print the history, triggers in this firmware to write the current temperature of the three temperature sensors in the LM32-log system.
• CONFIG_STILL_ALIVE_SIGNAL If this compiler switch defined, then the above mentioned animated software fan becomes compiled.

Location on ASL of the latest versions of both LM32 applications to test

The latest versions of the LM32 applications with FreeRTOS (scu3_control_os.bin) and the clessical version without FreeRTOS (scu3_control.bin) can be found on ASL in the following folder:

/common/export/nfsinit/global/lm32-test-apps/

On a SCU the same folder linked by NFS can be found here:

/opt/nfsinit/global/lm32-test-apps/

DAQ- Linux layer

The linux library libscu_fg_feedback.a

This library is the layer between the FESA- application and the LM32- firmware which receives the raw data from ADDAC DAQs and MIL-DAQs.

The main tasks of this library are:

• Sending of control commands to the LM32 firmware.
• Receiving the raw data from ADDAC-DAQ channels where the associated function generator is activated.
• Assigning 2 ADDAC-DAQ channels each that are responsible for actual- and set- values for the relevant function generator.
• Receiving the raw data from MIL-DAQ.
• Converting of ADDAC-DAQ data and MIL-DAQ data into a unified format, so that the overlying software layers doesn't care from what a kind of DAQ it trades.

If you works on the ASL then the documentation of the source code of the library libscu_fg_feedback.a, generated by Doxygen can be found in this folder:

/common/usr/cscofe/doc/scu/daq/scu_fg_feedback/html/

...and the following link should work: file:///common/usr/cscofe/doc/scu/daq/scu_fg_feedback/html/index.html
otherwise it's a dead link.

Building the static library libscu_fg_feedback.a

CAUTION: Prerequisite is that the LM32-binary file "scu3_control.bin" is already built, because this generates some header files which will be needed for this build process.

1. Change in directory gsi_scu/prj/scu-control/daq/linux/feedback/.
2. Set some environment variables for building by YOCTO-SDK:
   unset LD_LIBRARY_PATH; source /common/usr/embedded/yocto/fesa/sdk/environment-setup-core2-64-ffos-linux
3. Build binary file by typing: make.
4. If the binary was successful built it will be in this directory: scu-control/daq/linux/feedback/deploy_x86_64_sdk_4.1.1/result/libscu_fg_feedback.a
5. By the help of the script public.sh, which is in the same directory like the makefile, the new built library libscu_fg_feedback.a and all header files belonging to this library and the LM32- binary file scu3_control.bin,
   will copied in this directory: /common/usr/cscofe/opt/daq-fg/${FW_VERSION}${VERSION_DIR}/, The variable FW_VERSION becomes automatically read out from the LM32 binary file scu3_control.bin.
   Prerequisite for this script is that you works on the ASL.

C++ Classes of the DAQ-Library libscu_fg_feedback

The user-interface of both DAQ-types (MIL-DAQ and ADDAC-DAQ) is represented by three abstract C++ classes which are defined in the header file scu_fg_feedback.hpp:

1. FgFeedbackChannel
2. FgFeedbackDevice
3. FgFeedbackAdministration

At this abstraction layer it doesn't matter by witch type(s) of function generator(s) and DAQ(s) it trates. It can be the function generators and DAQs of MIL-devices or ADDAC- devices.

Class FgFeedbackChannel

An object of a class which inherits from the class FeedbackChannel represents the two DAQ-channels for set and actual values of the related function generator.

The virtual FgFeedbackChannel::onData() has to be overwritten by your own function.
This function becomes invoked by the function Administration::distributeDdata() ones a the following data of a function generator are available:

1. White rabbit timestamp.
2. The actual value from the ADC.
3. The set value of the function generator

This data set can be evaluated or forwarded in your function MyFeedbackChannel::onData().

The prototype declared in scu_fg_feedback.hpp is here:

virtual void onData( uint64_t wrTimeStampTAI, DAQ_T actValue, DAQ_T setValue ) = 0;

Class FgFeedbackDevice

A object of type FgFeedbackDevice is the container of at least one or more objects of type FgFeedbackChannel. It represents for example a SCU- bus slave. E.g. ADDAC, MIL (SIO or Piggy) or ACU device.
Each channel object of class FgFeedbackChannel respectively its driven class has to be registered by the function FgFeedbackDevice::registerChannel().

Class FgFeedbackAdministration

A object of class FgFeedbackAdministration is the container of at least one or ore objects of class FgFeedbackDevice. It represents the whole hardware of a SCU.
Each device object of class FgFeedbackDevice has to be registered by the function FgFeedbackAdministration::registerDevice().
This class contains the polling function FgFeedbackAdministration::distributeData(). This function has to be put in a polling-loop, which runs for example in a own thread.

Example for using the feedback classes

An code example how using this mentioned three classes can be found in the file feedback-example.cpp.

DAQ-helper tool "fg-feedback"

The tool fg-feedback based on the library libscu_fg_feedback.a. Therefore this tool is also the test- application for this library.
This tool can run directly on the SCU or on the ASL cluster or on a local Linux- PC which is within the GSI net.
If this tool runs not on a SCU, then a graphical output via GNUPLOT of the analogue set- and actual values of each activated function-generatur (MIL or ADAC) becomes possible like in the figure below, otherwhise only the raw data becomes displayed.

This figure above shows the plot of set- and actual values form the activated function generator fg-3-1. In this setup the analogue output of the DA- converter is directly connected with the analogue input of AD- converter therefore the graph of the actual-value (green) overlapped the set-value (red).
Furthermore this tool is very extensive and is constantly being expanded.

If this tool runns directly on a SCU or with the option -N then a probe of the probe of set and actual values with the number of incomming tuples for each function generator becones printet in text form like the figure below:

If one or more function generators has stopped its work, so this line becomes displayed in red, else it becomes displayed in green.

For more information about this tool, read the built-in help by typing:

fg-feedback -h

Wrapper Script "feedback.sh"

Because of the sometimes complicated commandline of fg-feedback there is a wrapper script feedback.sh for the most popular test configuration. For example the call

feedback.sh scuxl4711

leads to the figure above. The set- and actual values of each activated funktion-generator becomes plotet in a own gnuplot window.

Automatically allocation of DAQ channels for set- and actual values to the concerning function generator

As mentioned above when a function generator becomes enabled by the SaftLib then both belonging DAQ-channels for set- and actual-value becomes automatically enabled too.
Unfortunately is the allocation of DAQ- channels to the function generator not generally determined, so it differs between ADDAC, ACU and DIOB.
How does the application know which DAQ channel belongs to the set- and which DAQ channel to the actual value?
For this reason becomes a DAQ-FG- allocator necessary which is written in C and is linked to both applications: the linux interface libscu_fg_feedback.a and in the LM32 application scu3_control.bin or scu3_control_os.bin.
This allocator is implemented in the file daq_fg_allocator.c.

The determining code snippet of the allocation for the set-value is here:

unsigned int daqGetSetDaqNumberOfFg( const unsigned int fgNum, const DAQ_DEVICE_TYP_T type )
{
   DAQ_FG_ASSERT( fgNum < MAX_FG_PER_SLAVE );
   switch( type )
   {
      case DIOB: /* CAUTION: DIOB is a workaround! No break here. */
      case ADDAC:
      { /*
         * Returning of DAQ-channel number of ADDAC function generators
         * for set-values.
         */
         return fgNum + MAX_FG_PER_SLAVE;
      }
      case ACU:
      { /*
         * Returning of DAQ-channel number of ACU function generators
         * for set-values.
         */
         return fgNum * 2;
      }
      //TODO  DIOB here!
      default:
      {
         DAQ_FG_ASSERT( false );
         break;
      }
   }
   /*
    * Shall never be reached!
    */
   return 0;
}

And for the allocation of the actual value:

unsigned int daqGetActualDaqNumberOfFg( const unsigned int fgNum, const DAQ_DEVICE_TYP_T type )
{
   DAQ_FG_ASSERT( fgNum < MAX_FG_PER_SLAVE );
   switch( type )
   {
      case DIOB: /* CAUTION: DIOB is a workaround! No break here. */
      case ADDAC:
      { /*
         * Returning of DAQ-channel number of ADDAC function generators
         * for actual-values.
         */
         return fgNum;
      }
      case ACU:
      { /*
         * Returning of DAQ-channel number of ACU function generators
         * for actual-values.
         */
         return fgNum * 2 + 1;
      }
      //TODO  DIOB here!
      default:
      {
         DAQ_FG_ASSERT( false );
         break;
      }
   }
   /*
    * Shall never be reached!
    */
   return 0;
}

CAUTION: If in this code was made any changes then it becomes necessary that both applications linux and LM32 has bo be recompiled.

TODO List

• The shared memory section of the polynomial data for the function generators sent by the SaftLib shall be moved from the LM32 working memory to the SCU-RAM (DDR3 on SCU3 and in the future SRAM on SCU4).
  Political problem: This also makes it necessary to customize the SaftLib.
  • In this way a lot of LM32 working memory becomes free for further LM32 functions respectively tasks.
  • Because of much larger memory space becomes it possible to store a complete period of polynomial data independently for each function generator, which significantly reduces the load on the SaftLib and the bridge to the wishbone bus. At the moment this is not possible because the buffer in the LM32 working memory is far too small for this, so the SaftLib does permanently deliver the small LM32-memory with polynomial data. For example it will not be a problem to allocate 1 MB in the DDR3-RAM (respectively SRAM on SCU4) per function- generator.
• Unifying of the command receiving module respectively the algorithm in the LM32 firmware for DAQ- commands sent from the linux DAQ layer and te MSI- commands for the FGs sent from the SaftLib. (MSI commands becomes not necessary any longer).
  Political problem: This also makes it necessary to customize the SaftLib.
  • By this manner the LM32 firmware becomes simplified and will be more friendly to maintain and more sustainable.
  • And more LM32 working memory becomes free for new functions.
• When a ADDAC function generator becomes switched on by the SaftLib, then two ADDAC-DAQ channels, one for set-values and the other for actual values, which belongs to this function generator becomes automatically switched on as well. For test purposes it could be meaningful to disable this automatism. At the time this is impossible yet.
• Limitation of a rescan initiated by the SaftLib on the MIL- function generators only. At the time the whole SCU is involved in a rescan what is not meaningful because the SCU-bus slaves can be anyway exchanged only in the powerless state.
• Customizing on SCU4. That implies the new LM32- binary scu4_control.bin respectively scu4_control_os.bin.
  • The SCU4 will not have the MIL-PIGGY connector, that means unbundling of the LM32- firmware between MIL-SIO and MIL-PIGGY.
  • Instead of DDR3-RAM the SCU4 will have SRAM. That implies a new firmware module for SRAM for linux and LM32.
• Handling of Post-Mortem. The LM32 firmware is already able for that, but there is a lot of work for the extension of the linux layers.
• Fifo- administration of ADDAC- and MIL- DAQ has to move completely form LM32- shared memory section in the DDR3-RAM of SCU3 respectively in the SRAM of SCU4. This unravels the firmware and has already proven itself in the LM32 log system.

TODO