Contacts

• Ralph Steinhagen (Initial Project creator)
• Alexander Schwinn (FESA + build dependencies)

Overview

• Up to 4 picoscopes will be connected via USB to a server
• USB3 Ports are preferred to use higher sampling rates
• Overall in the end it is planned to have ~300 picoscopes
• Link to common spec (edms)
• Instruction how to start and use the class: StartDigitizerFESA
• Digitizer FAQ
• Related Group-Drive Folders
  • smb://srvfilegrp04.gsi.de/acogroup/Projects/Digitizer-Systeme/
  • smb://srvfilegrp04.gsi.de/acogroup/Group_HEL/PROJEKTE/Data Aquisition/Digitizer-Picoscope/
    • Open thunar on asl-cluster, paste the string into the address-bar, Domain: CAMPUS, Windows username/password
• Initial setup of a new SuperMicro server
• How to modify a grc file, which is used by the Digitizer Class
• Digitizer Expert Application, written by Benjamin Peter (link only works in acc-network)

Software stack

Sources:

• Gitlab repo Class
• Gitlab repo DU
• Gitlab repo gr-digitizers
• Gitlab repo gr-flowgraph
• DAQ API repo (Unified API for channel based aquisition), which is implemented by the Digitizer Fesa Class
• cabad repo (Library used to format and package data in the way suitable for DAQ API)

Simple picoscope binary to test low level picoscope drivers

• Sources to test ps3000(a?) are attached

gunzip ps3000Acon_Linux.zip
cd ps3000Acon_Linux
export CFLAGS=-I$HOME/picoscope
./autogen.sh
./configure
make
./ps3000acon

Known Issues

• Problem: Console Output: WARN: X Digitizer Data Buffers Lost
  • Solution: Too much CPU-Load. Either decrease sampling Rate, or throw away some Blocks ( or reconfigure them to cause less load )

• Problem: The Operateur wants to use a different gnuradio-file ( E.g. some fixes )
  • Solution:
    • Currently it is not possible to exchange a .grc file on "Rufbereitschaft" .. to many things which may go wrong. Wait until A.Schwinn is back, or if he is not @GSI, write him a mail.
    • For the future there will be a property to upload new *.grc files

Build & install gr-digitizers

cd gr-digitizers/build
cmake .. -DENABLE_GR_LOG=1 -DENABLE_STATIC_LIBS=ON -DCMAKE_INSTALL_PREFIX=/common/usr/cscofe/opt/gr-digitizer/master   (Use a different version than master if required !)
make -j8
./lib/test_digitizers_test.sh //runs unit tests
make install

For debug output add the following to cmake: -DDEBUG_ENABLED=1

Now you may want to provide the new lib for FEC's with:

cd digitizer2DU/scripts
./releaseDigitizer.sh dal12345

Build & install gr-flowgraph

export DIGITIZERS_DIR=/common/usr/cscofe/opt/gr-digitizer/master   (Use a different version than master if required !)
cd gr-flowgraph/build
cmake .. -DENABLE_GR_LOG=1 -DENABLE_STATIC_LIBS=ON -DCMAKE_INSTALL_PREFIX=/common/usr/cscofe/opt/gr-flowgraph/master   (Use a different version than master if required !)
make -j8
./lib/test_flowgraph_test.sh //runs unit tests
make install

For debug output add the following to cmake: -DDEBUG_ENABLED=1

Now you may want to provide the new lib for FEC's with:

cd digitizer2DU/scripts
./releaseDigitizer.sh dal12345

Configure Gnuradio Companion to run self-compiled Blocks

mkdir ~/.gnuradio
cd  ~/.gnuradio
touch config.conf

Add the following:

[grc]
local_blocks_path=/common/usr/cscofe/opt/gr-digitizer/master (Use a different version than master if required !)

Start gnuradio-companion:

gnuradio-companion

Script for FEC boot

Whenever the cluster gets a new gcc / new packages, it is required to re-deploy all digitizer dependencies. This can be done like that:

cd gr-digitizers
./createTarball_dependencies.sh

This will install all dependencies on a global nfs folder, where each Digitizer FEC can obtain them.

The nfsinits-script "01_digitizer_local" will extract all libs on reboot/when executed. On the asl development cluster it is located at:

asl744:/common/export/nfsinit/global/digitizer_local

E.g. on dal004 the script is linked like that:

asl744:/common/export/nfsinit/dal004/01_digitizer_local -> ../global/digitizer_local

The nfsinit scripts are versioned here.

Build FESA Class

If you want to run unit-tests, you currently need to use a local build environment (To be fixed with next release of the build environment)

Build FESA DU

Currently using a local-compiled version of fesa-fwk (fesa currently fails to take into account "firstupdate.acquisition = false" from fesa.cfg ) (bugfix should be available in next official fesa release)

Instance file:

• dal007 is curently running fine. Copy Instance file values from there
• For Timing simulation, make sure that GID matches simulated group (timingDomain= Whatever#1 )
• hwAddressMapping = {} is fine for a single scope (default is to use address from .grc -file )
  • For two digitizers one could e.g. write: {[digitizers_picoscope_3000a_0], [FP635/009], [digitizers_picoscope_3000a_1], [FP635/114]}
  • That is the digitizer with the associated block id digitizers_picoscope_3000a_0 will get assigned serial number FP635/009.
• Put correct IO for ftrnIoName (usually IO3 ) (Name of the timing receiver output )
• Always use the Ring timing domain in FESA (e.g."SIS18_RING#300" or "CRYRING_RING#210" )

Export new build version to FEC

Inside DigitizerDU2 a script "releaseDigitizer.sh" can be used to copy all relevant stuff on a FEC. Add the FEC to release as parameter. E.g:

./releaseDigitizer.sh dal004

Add new Digitizer Server and Deploy Software on it

• Usually IN notifies via email if a new server was installed
• Open the list of all Digitizer Systems: Digitizer_MasterPlan.odt
  • Add the new system to the correct Device and fill in Missing Info
  • Mark each row "green" which you prooved to work
• Setup for devacc tools in .bash_profile on asl cluster. Required!
  • see cdi /cdf for quick navigation to init folder / fesa folder

######## Setup for devacc tools ###########
. /common/usr/cscofe/scripts/accdefs

• get Digitizer sources:
  • git clone https://gitlab.com/al.schwinn/DigitizerClass2.git
  • git clone https://gitlab.com/al.schwinn/DigitizerDU2.git
  • "git checkout alpha8" in both projects
• Execute DigitizerDU2/scripts/addDigitizerFEC.sh -- This will automatically:
  • Setup pxe links, to provide the right image for the FEC (version alpha8 still needs doomsday)
  • Create Init folder. E.g: mkdir /common/export/nfsinit/fel0049 and create symlinks there
  • Create Fesa folder. E.g: mkdir /common/export/fesa/local/fel0049 and create symlinks there
  • Install most recent gateware
• Go to FEC: sss fel0049
  • Check if gateware has the correct version (same as linked Timing lib in FEC Init folder)
  • Check if FEC has link
    • If not, Ask Timing Group if e.g. cable is missing (usually Markus Zweig)
  • Send a mail to Timing Group (csco-tg@gsi.de) to inform that the Pexaria is now in use
    • Required Information: FEC-Name, Room, Serverrack, Responsible, Pexaria MAC-Address
    • Get the MAC-Address of the pexaria with: "eb-mon dev/wbm0 -v"
    • Note: when you are done with installing stuff, shutdown the FEC until it got registered by Timing (command: poweroff)
• install Eclipse Fesa plugin: https://www-acc.gsi.de/wiki/FESA/FESA3Install400
• build DigitizerClass/DU:
  • In eclipse "File -> Import --> Projects from git" Import both projects
  • rightclick on DigitizerClass2 --> Fesa --> Synchronize Source Code (generates c++ code)
  • rightclick on DigitizerDU2 --> Fesa --> Synchronize Source Code (generates c++ code)
  • Go to "make targets" in eclipse, doubleclick "all x86_64", first on class, than on DU --> now you should have a binary in DigitizerDU2 /build/bin
• Configure frontend:
  • Rightckilck on Project DigitizerDU2 --> FESA --> AddFEC --> clone existsing instance file (e.g. clone a FEC which uses the same scope
  • Copy *.grc file to new folder by hand
  • Fix the path of the *.grc file in DeviceData _DigitizerDU2.instance
  • Fix the attribute "name" of the "device-instance" in DeviceData_DigitizerDU2.instance (Column "FESA Device Nomen" in the Digitizer_MasterPlan.odt)
  • Set gnuradio block search path: https://www-acc.gsi.de/wiki/FESA/FESA-Class-Digitizer#Configure_Gnuradio_Companion_to_run_self_45compiled_Blocks
  • Open gnuradio-companion <*.grc> (e.g. doubleclick in eclipse on *.grc file)
  • Edit Blocks, according to your needs .. on multiple scopes, you as well need to provide the scope serial-number (On Rapid Block mode, use TimeSinks, on Streamng mode, use cascade sinks)
  • Edit siganlname, cquisition mode (streaming/rapidBlock), samp. rate, trigger_samples, scaling, offset : Ask Ralph Steinhagen what is required.
• Release with release script: DigitizerDU2 /scripts/releaseDigitizer.sh : ./releaseDigitizer.sh fel0049
• Go to FEC: sss fel0049
• Go to Fesa folder on FEC: cdf; cd DigitizerDU-d
• Start Fesa Software by hand: startManually_DigitizerDU2_M.sh .. see if it runs. Fix bugs according to errors shwon, if it does not run. (Exit with CTRL+C)
• Create a zip file im eclipse plugin for Fesa Explorer : Open DeviceData _DigitizerDU2.instance, "Launch Fesa Explorer"
• Start Fesa Explorer ... here my Launcher: ssh -X asl744.acc.gsi.de 'fex --vmargs="-Dcmw.directory.client.serverList=cmwpro00a.acc.gsi.de:5021"'
• In FEX on top left conner, open --> select the generated zip file
• Further steps are explained here: https://www-acc.gsi.de/wiki/FESA/StartDigitizerFESA#Usage_of_a_Digitizer_Instance_via_Fesa_Explorer
• If you see some Signal / Noise, than its time to start the Software in daemon mode:
  • In FESA NFS folder on the asl (cdf fel0049) create a symlink: ln -s /common/export/fesa/local/fel0049/DigitizerDU2-d /common/export/fesa/local/fel0049/DigitizerDU2
  • In Eclipse Plugin rightclick on projekt DigitizerDU2 --> Release --> Environment: InUse, select the relevant FEC
  • On FEC: cdi ; ./50_fesa
    • That will start a deamon for each Fesa binary which has the same name than the containing folder (Haha, bad design)
• Export the Instance file to the Database by opening "DeviceData_DigitizerDU2.instance" in eclipse and pressing "Export FESA Insatnce to the Database" ... (only like that, Ben will be able to communicate with the Software via Japc)
• If stuff runs well, inform Ralph Steinhagen and Benjamin Peter, and Alexander Schwinn.
• Push changes to release branch and cherry-pick them into master

WR Timing

Wenn der server nicht an einen WR-Switch angeschlossen ist muss den TR in "master mode" versetzen um Timing simulieren zu können ( sonst gibt 'not locked' Fehler )

Achtung: Kein Timing Kabel anschließen wenn in master mode !! Das könnte evtl zu falschen Events im Timing-Netz führen.

Versetzen in master mode:

eb-console dev/wbm0
ptp stop
mode master
CTRL+C

Zurück in den normalen modus geht so:

eb-console dev/wbm0
ptp stop
mode slave
CTRL+C

Optional kann man in der eb-console auch die Zeit einstellen (per default startet die Zeit bei 0)

time set <sec> <nsec>

Die UTC clock des systems in sekunden bekommt man so:

date +%s

Ein einzelnes WR Timing event simulieren: saft-ctl tr0 inject 6 0 0

Erzeugen einer 20Hz Rechteckspannung auf IO2 der Pexaria:

saft-io-ctl tr0 -n IO2 -t 0 -o 1
saft-clk-gen tr0 -n IO2 -f 20 0 -v

Hier gibt es verschiedene script zur simulation von Timing events

Hier gibt es andere nützliche Scripte, z.B. zum releasen von Digitizer Software.

Script zum releasen aller Komponenten: DigitizerTestDU2/releaseDigitizer.sh