WR-MIL gateway Introduction

WR-MIL gateways are a replacement for the existing SIS and ESR Pulszentrale (PZ). They are implemented in software running on a LM32 softcore CPU inside a SCU. One WR-MIL gateway is needed to replace the SIS18-PZ, another one is needed to replace the ESR-PZ. The WR-MIL gateway reacts on a WR timing event if the groupID is correct (0x12c for SIS18 and 0x154 for ESR) and the eventID.EVTNO is in the range of 0...255 by emitting a MIL timing event (or MIL telegram) on the SCU device bus connector as described here. The bits of the MIL event are a combination of the eventID.EVTNO[7:0] (MIL[7:0]) and eventID.SID[7:0] (MIL[15:8]), i.e. the "event number" and the "sequence-ID" of the WR timing Messages. One special MIL timing event (configurable, default EVT_END_CYCLE) triggers the generation of the following sequence:

• <the trigger event itself>
• EVT_UTC_1
• EVT_UTC_2
• EVT_UTC_3
• EVT_UTC_4
• EVT_UTC_5

If the gateway is not getting WR-events that can be converted to MIL, it starts generating an EVT_INTERNAL_FILL MIL-event every ~9 seconds.

The device bus cable goes from the SCU into a converter/splitter box from where the MIL event bus (lemo) cables carry the MIL timing event to the MIL timing receiver modules (for example SE, TIF).

Figure 1: Connections between SCU and converter box.

Some remarks

• Sending a MIL event takes ~20 us. MIL events are sent sequentially. Therefore, each MIL event causes a dead time of the gateway of ~20 us. If a WR timing message for the gateway has a deadline that is less than 20 us after the previous message's deadline, the MIL telegram will be delayed.
• If an event number triggered the generation of EVT_UTC_1/2/3/4/5, the gateway is busy for a longer time (~180 us).
• The MIL events have an approx. ±3us jitter.

Diagnostics

• The Wr-MIL gateway SCUs send log messages into graylog.
  • for SIS18 gateway
  • for ESR gateway
• A message appears in the following cases:
  • The gateway is started, stopped, or reconfigured (to SIS or ESR)
  • The gateway gets/looses WR-lock
  • The gateway's OP_READY state changes
  • A MIL event couldn't be sent on time (a late MIL event)
  • No WR-event was converted to MIL since more than ~9 seconds. In this case the Gateway goes into IDLE-state and starts generating EVT_INTERNAL_FILL MIL-events
  • The gateway was in IDLE state and receives translates WR-events again.

Configuration

Any SCU can be configured as WR-MIL gateway by flashing the correct gateware and creating a symbolic link to the correct timing-rte.

Gateware

The gateware version for the WR-MIL gateway SCUs is standard enigma

Symbolic links

For the SIS18 gateway (scuxl0228):

[mreese@asl740 scuxl0228]$ pwd
/common/export/nfsinit/scuxl0228
[mreese@asl740 scuxl0228]$ ls -lh
total 0
lrwxrwxrwx 1 mreese bel 32 May  3  2018 20_timing-rte -> ../global/timing-rte-tg-wrmilgw-SIS18

For the ESR gateway (scuxl0068):

[mreese@asl740 scuxl0068]$ pwd
/common/export/nfsinit/scuxl0068
[mreese@asl740 scuxl0068]$ ls -lh
total 0
lrwxrwxrwx 1 mreese bel 32 May  4  2018 20_timing-rte -> ../global/timing-rte-tg-wrmilgw-ESR

Deployment

• On the asl cluster
• git clone https://github.com/GSI-CS-CO/ci_cd.git
• cd ci_cd/scripts/deployment/RTE_Timing
• edit build-rte.sh header to look like this (example is for SIS18, for ESR make the obvious changes)

#PLEASE ADJUST THIS SCRIPT FOR YOUR NEED
BEL_BRANCH="wr-mil-gw-saftlib"
BEL_RELEASE=""
BEL_BUILD_ADMIN="no"
BEL_BUILD_WRMILGW="SIS18"  # can be "SIS18" or "ESR" to activate the gateway
BEL_BUILD_KERNEL4="no" # build for kernel 4+
#Targets for the TG: R8-balloon_0 RC8-balloon_0 tg-dev tg-testing
#For the rest of the Groups, you can create one for your need
DEPLOY_TARGET="/common/export/timing-rte/tg-wrmilgw-SIS18"

• ./build-rte.sh

Location

Two SCUs in BG.2.009c

• The WR-MIL gateway for SIS18 runs on scuxl0228
• The WR-MIL-gateway for ESR runs on scuxl0068

Bugs

During the beamtime 2021, several reports of missing MIL events appeared. For example, the electron cooler in ESR was sometimes not switched off because of a missing event. The reason for the missing events is not found yet. This is a collection of observations related to the missing events. It may help in finding the cause of the probelm.

• The WR-event that corresponds to a missing MIL-event is definitely there. The events are not missing in the schedule of the Data Master.
• The conversion of a WR-event into MIL-event happens in an LM32 program. The LM32 sends MIL-events with the correct deadline into the ECA, these events have the ID=0xffffffff0000MMMM, where M represents the bits of the MIL-event. The ECA-WB-Master output channel writes the MIL-event into the MIL-Piggy. By configuring the ECA-IO channel to pulse one of the SCU-IOs at the same event, it can be seen that the firmware indeed injects the event into the ECA. However sometimes the MIL-Piggy doesn't send anything. The Missing Events can be observed on the oscilloscope.
• 
• It is unclear why this happens. Speculation: the MIL-Piggy doesn't accept that WB-Write from the EAC-WB-Master output channel for some reason, or the WB-Master output channel has a bug and doesn't execute the WB-write, or the MIL-Piggy accepts the WB-write but has a bug and doesn't send the MIL-Event. => Make a SignalTap that targets the WB-interfaces of ECA-WB-Master output channel and the WB-interface of the MIL-Piggy
• The MIL-Events of the prduction systems (SIS18 and ESR) in BG2.009 have glitches after the last Manchester-bit. This doesn't seem to be a problem because it always happens after the last bit (the parity bit). These glitches do not appear on a similar setup in my office. The reason/source of the glitches is not located yet.
• 
• The rate of missing events depends on the schedule, but approximately every 30-60 minutes a MIL event is lost. Often the missing MIL-events are not essential for machine control (e.g. missing UTC timestamp bits). Missing MIL events that are important appear a few times per day.