Fallout Release

Previous releases are Asterisk, Balloon, Cherry, Doomsday and Enigma

New features and bug-fixes in Fallout Release

  • 2020-Aug-31: [v6.0.0-alpha-rc1] alpha release candidate RC1
    • critical stability fixes and improvements
    • upgraded to new White Rabbit PTP Core v4.2
    • upgraded White Rabbit Switches to v6
    • new USB driver
    • saftlib via USB (see documentation on concurrent use of saftlib and Etherbone for USB)
    • timing constraints and closures fixed
    • LLDP for nodes
    • PEXP support
    • SCU4 support
    • OHWR merged -> "back on track"
    • EBM fix for latest general-cores (critical)
    • ECA-Tap for all form factors
    • LM32-Timer
  • 2020-Oct-01: [v6.0.0] release
    • minor stability fixes and improvements
    • dm-unipz ('SIS Anforderung') supports UNILAC beam preparation 'CMD_UNI_BPREP'
    • wr-unipz ('Pulszentrale') supports UNILAC service event 'EVT_UNLOCK_ALVAREZ'
    • WB-crossbar type attributes (Michael's patch)
  • 2020-Nov-03: [v6.0.1] release
    • White Rabbit core fixes (fast lock and smooth console)
    • Saftlib fixes (data race/thread-safety)
  • 2021-Aug-09: [v6.1.2] release
    • Etherbone v2.1.2 (dropping all unsolicited EB packets)
    • Saftlib v2.4.3
      • wr-mil-gw: minor improvements
      • saft-burst-ctl: official release
      • realtime improvements (relevant for function generator applications)
        • ECA is configured such that only software actions produce MSIs (previously all ECA-actions (IO,SCU-bus,...) produced MSIs).
        • Deprecated interfaces with uint64_t as time were removed (reduces number of saftlib signals)
        • no more dynamic memory allocations in fifo buffers.
      • debug messages into graylog when exceptions occur
      • other minor tweaks
    • MIL-Bus bugfix (needed by WR MIL gateway)
    • 1ns-IO bugfix (fixed glitches on high frequencies)
    • LM32 ECA queue access improvement (moved the slave from dev to top crossbar)
    • WR/SFP reset feature (eb-reset device phyreset, eb-reset device sfpreset)
    • B2B and common LM32 functions merge/update
    • Further Arria10 developments (designs and IP cores)
  • 2021-Nov-02: [v6.2.0] release
    • Saftlib v2.5.0
      • FG/MFG fixes
      • Fixed exceptions
      • Read combined output from IOs
    • hdlmake 3.3 support (Python2.7 and Python3.X)
    • new watchdog feature for SCUs with ACO-RAM-Disk and Timing RTE
      • watchdog is permanently active
      • in case the host system (or 'saftd') is dead, the watchdog will reset FPGA and COM Express board (host system) within 10 minutes
      • it is recommended to disable the watchdog when flashing an SCU with a new FPGA image
  • 2022-Mar-16: [v6.2.1] release
    • wb_mil_scu: added priority encoder + bug fixes (SCU2/SCU3/SIO3 gateware only)
  • 2022-Jul-15: due to missing support for NFSv3, the ramdisk 2021-10-26 must no longer be used; scuxl.fallout now points at ramdisk 2022-07-04.el7

Readme

Developers should read this.

Known Issues of Fallout Release

  • 2020-Sep-28: 32bit Raspbian - Timing Receiver connected via USB
    • saftlib works
    • Etherbone works
    • concurrent use of Etherbone and saftlib fails
    • this issue is not present for 64bit systems and other host system buses
  • 2020-Oct-13: customers from BI report instabilities of saftlib when removing conditions
  • 2020-Oct-14: customers from EEL report two issues
    • direct PCIe mapping does not work as expected
    • 'no lock' for some devices (edit: strange, but cured by replacement of WRS)
  • 2020-Nov-03:
    • issue with saftlib (fixed)
    • issues with WR core (fixed)
  • 2021-Jan-31:
    • in some cases there is an issue after power-up, when a WRS becomes operatational after the timing receiver
    • somtimes this shows up as no 'White Rabbit lock'
    • in rare cases even the physical Ethernet link is not established
    • usually 'eb-reset fpgareset' helps
      • if not, try 'eb-reset sfpreset'
  • 2021-Sep-23:
    • in release fallout-v6.1.2 there, the related saftlib version 2.4.3 has broken exceptions in saftlib driver functions. These exceptions are not directed correctly to the Proxy object. Workaround: use a different saftlib version (branch "fix-exceptions", git commit b57d8e473f4) which is identical to version 2.4.3 but has a bugfix for exceptions.
    • generating the saftlib pdf-documentation does not work out of the box. A precompiled pdf file for saftlibv.2.4.3 is located here.

Wishlist for Upcoming Minor Releases

Release Compatibility Matrix

Release Name Date Release Version Gateware PXE config Timing RTE Etherbone Saftlib Ramdisk Type GIT hash Remarks
Fallout RC1 2020-Aug-31 6.0.0-rc1 6.0.0-rc1, link scuxl.fallout timing-rte-tg-fallout-v6.0.0-alpha-rc1 2.1.1 2.2.2 2020-06-17 alpha release candidate e96bae2  
Fallout 6.0.0 2020-Oct-01 6.0.0 6.0.0, link scuxl.fallout timing-rte-tg-fallout-v6.0.0 2.1.1 2.2.4 2020-06-17 release ae8138f  
Fallout 6.0.1 2020-Nov-03 6.0.1 6.0.1, link scuxl.fallout timing-rte-tg-fallout-v6.0.1 2.1.1 2.2.5 2020-11-09 release 9aed415  
Fallout 6.1.2 2021-Aug-09 6.1.2 6.1.2, link scuxl.fallout timing-rte-tg-fallout-v6.1.2 2.1.2 2.4.3 2021-07-30 release 33b311a  
Fallout 6.2.0 2021-Nov-02 6.2.0 6.1.2, link scuxl.fallout timing-rte-tg-fallout-v6.2.0 2.1.2 2.5.0 2021-10-26 (-> 2022-07-04.el7) release d95c321  
Fallout 6.2.1 2021-Nov-02 6.2.1 6.2.1, link scuxl.fallout timing-rte-tg-fallout-v6.2.0 2.1.2 2.5.0 2021-10-26 (-> 2022-07-19.el7) release 46dc8fb Only for SCU2&3 and SIO3!
Table: Version matrix. Color coded are entries that require flashing (gateware) or changing symbolic links (PXE config, Timing RTE). Listed are current (green) and old (red) releases.

Just in Case: Nightly builds for Fallout gateware/firmware/software are available here.

Please consult the instructions for flashing.

Setting up Timing Receiver Nodes

Follow these steps

1. Gateware Image
  • please download gateware images 'rpd' by following the link in the table above
  • downloaded images must be written to a flash chip on each individual timiming receiver hardware; see here for flash instructions.

2. Ramdisk (ACO environment only)
  • make sure the host system is (via PXE) booting the ramdisk specified in the table above
  • information on the PXE boot process is described here
  • typically, you just need to create a symlink < HOSTNAME > -> < PXE-CONFIG >

3. Timing Runtime (RTE, ACO environment)
  • use the RTE configuration specified in the table above
  • information on nfs-init is described here
  • typically, you just need to create a symlink 20_timing-rte -> ../global/timing-rte-tg-fallout-v< VERSION >
  • other OS: follow the directions given in this how-to

4. Socat
  • socat has been disabeld
  • if you need socat
    • create a symlink 90_timing-socat-wbm0 -> ../global/timing-rte-tg-socat; try 'wbm0..3' in case your host contains multiple timing receivers
    • reboot
    • the use of socat is forbidden in the production environment

Checking Timing Receiver Nodes

Command Line

Follow these steps

1. Gateware Image Version
[ruth@scuxl0815 ~]# eb-mon -a dev/wbm0
fallout-v6.0.1

2. Ramdisk Version
[ruth@scuxl0815 ~]# cat /etc/os-release
GSI embedded release 7 (build 2020-11-09)

On 'yocto ramdisks', the information obtained via '/etc/osrelease' or 'uname -a' does not necessarily change when a new ramdisk is released. One can additionally check
[ruth@scuxl0815 ~]# cat /etc/image_manifest              // analyzing this might be tedious
  or
[ruth@scuxl0815 ~]# md5sum /etc/image_manifest           // this is unique but does not carry information

3. Timing Runtime Version
[ruth@scuxl0815 ~]# cat /etc/timing-rte_version
6.0.1

4. Timing Runtime Buildinfo
[ruth@scuxl0815 ~]# cat /etc/timing-rte_buildinfo
GSI Timing RTE 02-11-2020_13-34-55 
Compiled by ahahn using ./build-rte.sh on asl743.acc.gsi.de - Linux  3.10.0-1127.10.1.el7.x86_64
CI_CD Project
 - https://github.com/GSI-CS-CO/ci_cd.git
 -  master*@044103d
BEL_PROJECTS
 - https://github.com/GSI-CS-CO/bel_projects.git
 - fallout-v6.0.1@25bdabb
Last Commits in repo:
25bdabb saftlib: v2.2.5
07b076f Merge branch 'dm-fallout-merge-v2' of https://github.com/GSI-CS-CO/bel_projects into fallout
cdfa227 DM: tools: fixed possible buffer overflow due to long directory name in dm-cmd
2a0a570 README.md: updated
998ed87 Flag Beamprocess Chain start: implementation

Trouble? Follow the steps given here

Graylog

Use search string source:scuxl0815 AND message:trinfo. This yields diagnostic info generated at host system boot time.

1. FPGA Info 'Build type' shall match the compatibility matrix.

scuxl0815 timing: trinfo FPGA: FPGA model : Arria II GX (ep2agx125ef29c5) ; Platform : scu3 +comexpress ; Build type : fallout-v6.0.1 ; uptime [h] 0000000000.02
                                                                                                                                ^^^^^

2. White Rabbit Network Interface The MAC might is useful for registering your TR (in case you have not yet done so).

scuxl0815 timing: trinfo WR NIC: MAC: 00267b00ce42 ; IP: 192.168.47.11 ; Ethernet: LINK_UP ; WR-PTP: TRACKING

3. Timing RTE and Ramdisk 'RTE' and 'ramdisk build' shall match the compatibility matrix.

scuxl0815 timing: trinfo RTE: 6.0.1 @ramdisk: GSI embedded release 7 (build 2020-11-09)
                              ^^^^^                                         ^^^^^^^^^^

Supported Hardware

Form Factor Carrier WR add-on Commissioning Remark
standalone EXPLODER5A WREX1/WREX2A EXPLODER5ADB2 click
PCIe PEXARIA5XX WREX1 PEXARIA5DBYY click "fixed" PEXARIA5 boards are compatible
SCU2 SCU2 WREX1 (MIL)  
SCU3 SCU3 on-board (MIL)  
SCU4 SCU4 on-board    
VME VETAR2A WREX1 VETAR1DB2A click  
AMC tr-amc on-board N/A click
PMC tr-pmc on-board N/A click
PCIe PexP on-board N/A click
Table: Supported hardware.

Further Reading

Bitstream Types

Types of Bitstream Platform DescriptionSorted ascending How to Flash or Program
jic Altera JTAG Indirect Configuration File. This file contains the TR Gateware. It's used to program the FPGA. It is not persistent Quartus Programmer
rpd Altera Raw Programming Data File. This file contains the TR Gateware. It's used to write the gateware into the flash memory of the TR. It is persistent eb-flash
sof Altera SRAM Object File. This file contains the TR Gateware. It's used to write to program the FPGA. It is not persistent Quartus Programmer
jed Xilinx This file contains the special Gateware for the CPLD on the TR. It is persistent Xilinx Programmer

Data Master

Sources for Data Master images and software are part of the GIT repository in dedicated branches.

Getting the sources code of the Fallout Release from our GIT Repository

If you want to check the source code this Release is in the branch 'fallout' in bel_projects.

git clone git@github.com:GSI-CS-CO/bel_projects.git  // try 'git clone https://github.com/GSI-CS-CO/bel_projects' if not using public/private key authentification
cd bel_projects
git checkout fallout # or git checkout fallout-v6.x.y
make

More Features

Feature Exists/Status Included 6.1.N (later) Postponed
new WR 4.2 Exists X    
new WRS 6.0 Beta X    
new PCIe driver Unkown     X
new USB driver Exists X    
saftlib via USB Exists X    
include JAM/EE kernel driver patches (new PCIe driver) Unkown     X
bursts (for LOBI) Waiting for feedback     X
regain FPGA ram Needs development     X
timing constraints and closures Arria2 SDC problem X (Arria5 + Arria10)   X
clocking scheme cleanup Needs development     X
'UTC issue' UTC offset via WR Unkown, WR 4.3?     X
dm-unilac (generator integration)       X
dm-unilac (update and fixes) Pending   X  
wr-unilac (update and fixes) Pending   X  
update MASP emmitter (wr->mil, dm-unilac, ...)     X  
802.1x to be done   X  
LLDP (node) Exists X    
LLDP (WRS, tsl001, ...)     X  
timing groups for internal usage available     X  
timing event number of internal usage available     X  
concept "how to deploy and release custom lm32 firmware" (bursts, ...) Needs development     X
pexp make target Exists X    
scu4 make target Exists X    
OHWR merge -> "get back on track" Mostly done X    
AXGE-1254-0531/AXGE-1254-0531 calibration fixed delays values and alpha parameters Needs development     X
Crossbar redesign (seconds dev bar) Under test (TC)     X
EBM fix for latest general-cores (critical) Needs development X    
ECA-Tap for ALL Exists X    
LM32-Timer (UBOS) Exists X X  
Crossbar type attributes (Michael's Patch) Exists X    
Saft-Feet Exists     X
Dirty-flag tool-chain (MK?)     X  
SCU Watchdog 2.0 (SR)     X  
LVDS/Serdes patch (AH) Exists   X  
New/same flash controller for all devices (SR/AH(testing)) Under development   X  
Burst-Firmware Target/Release Exists   X  
Table: Features, status and timeline. Experimental features are marked by round brackets.

More Information

  • building and deployment the Timing Run-Time Environment, see here
  • building and deployment of software and drivers for various types of linux boxes (including SCU), see here and here
  • flashing timing receivers with new images (including SCU), see here
  • some hints for FECs, see here

-- DietrichBeck, AlexanderHahn - 14 March 2023
Topic revision: r56 - 26 Apr 2023, DietrichBeck
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback