Realtime Communication

Table of Contents

Introduction

The b2b system is a distributed system where distinct components are required to communicate in hard realtime. This can not be done with unicast traffic. First, White Rabbit switches are optimized for broadcast traffic. Second, if broadcast would hurt other realtime traffic, so would unicast. Broadcast is done by sending timing messages to the ECA address, see protocol.

Timing messages are sent ahead of time. As an estimate of upper bound latency, propagation through the network requires 500 us. The b2b system requires a series of three messages (3 x 500 us) plus another 500 us of processing time in the nodes. As an estimate, the b2b system requires 2 ms before kickers can be triggered. The b2b system starts its activity already 500 us prior to the beginning of the flat top. The earliest time of the kicker trigger is 1500 us after the beginning of the flattop. The b2b system offers dedicated diagnostics to monitor its realtime behavior.

Remark: For simplicity, the time differences in the figures on this page assume that the ParamModi parameter B2B Trigger Verschiebung is set to 0 degree. Detailed information on kicker timing is given here.

The precise message form is described here.

Messages for Different Modes

This section describes the situation for SIS18 and ESR.

The situation at CRYRING is slightly different. At CRYRING

• the message CMD_B2B_START is scheduled 3.0 ms after the beginning of the extraction flattop
• the message CMD_BEAM_EXTRACCTION is scheduled ~ 5.0 ms after the beginning of the extraction flattop

Mode EKS (EVT_KICK_START)

Figure: Communication for mode EKS. Time propagates from left to right (not to scale). Horizontal line depict the components Data Master (DM), Central Bunch-2-Bucket Unit (CBU) and Kicker with Diagnostic of the extraction ring (KD_EXT). Shown as vertical lines are messages CMD_B2B_START (B2BS), CMD_BEAM_EXTRACTION (BE), EVT_KICK_START (KS), CMD_B2B_TRIGGEREXT (KTE) and CMD_B2B_DIAGKICKEXT (KDE). The line style depicts messages deadlines (solid, green, top), time of messages reception (dashed, green, bottom) and time when messages are sent (solid, magenta, top). The blue arrows help to guide the eye. The yellow box indicates the length of the extraction flattop. In mode EKS, the b2b system triggers the kicker 1500 us after the beginning of the extraction flattop. See text for details.

This is the simplest mode provided by the b2b system. The process is started when the CBU receives the message B2BS from the Data Master. First, the CBU will 'commit' the set-values from the FESA class; the set-values are checked for consistency copied from the Dual Ported RAM to internal RAM. Second, the CBU sends out the trigger event for the extraction kicker using a deadline of 1500 us after the beginning of the extraction flattop. At the extraction kicker, the component Kick and Diagnotic (KD_EXT) will trigger the kicker electronics and acquired data, see here. When this is done, the KD_EXT will encode and publish the data by the timing message CMD_B2B_DIAGKICKEXT (KDE). KDE has a deadline of KTE + 1 ms.

Additionally shown are the deadlines of two events that are sent by Data Master. CMD_BEAM_EXTRACTION (BE) is scheduled at the beginning of the extraction flattop and EVT_KICK_START1/2 (KS) is scheduled a bit sooner than the earliest deadline of KTE. These two events are not relevant for the b2b system.

This mode requires the following messages at the b2b system

• CMD_B2B_START (B2BS)
• CMD_B2B_TRIGGEREXT (KTE)
• CMD_B2B_DIAGKICKEXT (KDE)

Mode B2E (Bunch to Extraction)

Figure: Communication for mode B2E. See caption of figure above for basic explanation. Additionally shown here is the component Phase Measurement of the extraction ring (PM_EXT) and the messages CMD_B2B_PMEXT (PME), CMD_B2B_PREXT (PRE) as well as CMD_B2B_DIAGEXT (PDE). The green boxes indicate a time span withing the extraction kicker is triggered. Details see text.

This mode is used for the so-called 'fast extraction', usually to a cave or a fixed target. The process is started when the CBU receives the message B2BS from the Data Master. Then, the CBU sends out the message CMD_B2B_PMEXT (PME) with a deadline that coincides with the beginning of the flattop. PME is received by the Phase Measurement of the extraction ring (PM_EXT). After the phase measurement is completed, PM_EXT sends the result via the message CMD_B2B_PREXT (PRE) with a deadline of PME + 500 us. Typically, this is done 100 us after the beginning of the flattop. For saving time, the CBU is has configured its ECA with a negative offset of 250 us for PRE. The CBU will then do some math and schedule the message CMD_B2B_DIAGKICKEXT (KTE) at the next rising edge of the h=1 group DDS following two full millisecond after B2BS (1.5 ms after BE). Thus, the extraction kicker is triggered in a certain time span depicted by green boxes in the figure above. For mode B2E, the length of the green box equals the revolution time of the beam in the extraction ring.

As an additional information, the b2b system records some values of signal propagation which is important to diagnose malfunctions of the b2b system and potential violations of upper bound latency:

• t0E-CBS: time difference of the phase measurement (extraction) to B2BS. This is in the order of 600 us.
• fin-CBS: time difference of 'the end' of the CBU activity to CBS for a given transfer. For fast extraction, this is expected to be in the order of 800 us.
• KTE-fin: time difference of KTE to 'the end' of the CBU activity. For fast extraction, this is a expected to be in the order of 1200 us. This value must not be less than 500 us.
• KTE-CBS: time difference of KTE to CBS. This is expected to be 2.0 ms plus 'x', where 'x' is in a window between 0 and the RF revolution time in the ring.

Furthermore, the b2b System acquires and publishes diagnostic data.

• KD_EXT measures signals of kicker electronic and magnet probe, see here. This data is published via the timing message CMD_B2B_DIAGKICKEXT (KDE).
• PM_EXT measures two signals to verify correct kicker timing and the RF system is working as expected. This data is published via the timing message CMD_B2B_DIAGEXT (KDE).

This mode requires the following messages.

• CMD_B2B_START (B2BS)
• CMD_B2B_PMEXT (PME)
• CMD_B2B_PREXT (PRE)
• CMD_B2B_TRIGGEREXT (KTE)
• CMD_B2B_DIAGKICKEXT (KDE)
• CMD_B2B_DIAGEXT (PDE)

Mode B2C (Bunch to Coasting Beam)

Figure: Communication for mode B2C. See caption of the figures above for basic explanation. Additionally shown here is the Kicker and Diagnostics at the injection machine (KD_INJ) and the messages CMD_B2B_TRIGGERINJ (KTI) and CMD_B2B_DIAGKICKINJ (KDI). Details see text.

This is the simplest mode for transferring beam between two rings. Extraction happens 'rf-triggered' but without synchronization to the RF of the injection machine. This is basically the same as fast extraction but with an additional trigger for the kicker of the injection machine. Here, the b2b component Kick and Diagnostics for injection (KD_INJ) receives a dedicated timing message CMD_B2B_TRIGGERINJ (KTI) and publishes diagnostic information via the timing messages CMD_B2B_DIAGKICKINJ (KDI).

As an additional information, the b2b system records some values of signal propagation which is important to diagnose malfunctions of the b2b system and potential violations of upper bound latency: * KTI-CBS: time difference of KTI to CBS. This is expected to be 2.0 ms plus 'x', where 'x' is in a window between 0 and the RF revolution time in the ring.

This mode requires the following messages.

• CMD_B2B_START (B2BS)
• CMD_B2B_PMEXT (PME)
• CMD_B2B_PREXT (PRE)
• CMD_B2B_TRIGGEREXT (KTE)
• CMD_B2B_DIAGKICKEXT (KDE)
• CMD_B2B_TRIGGERINJ (KTI)
• CMD_B2B_DIAGKICKINJ (KDI)
• CMD_B2B_DIAGEXT (PDE)

Mode B2B (Bunch to Bucket)

Figure: Communication for mode B2B. See caption of figure above for basic explanation. Additionally shown here is the component Phase Measurement of the injection ring (PM_INJ) and the messages CMD_B2B_PMINJ (PMI), CMD_B2B_PRINJ (PRI) as well as CMD_B2B_DIAGINJ (PDI). See text for details.

This is the mode for true bunch-to-bucket transfer. As a difference to mode bunch-to-coasting beam, the communication with module Phase Measurement of the injection ring (PM_INJ) and three timing messages are added. Moreover, this mode is the most complex one for the CBU, which needs to collect and combine data from various sources. The CBU also requires more time dedicated to math. Presently (December 2021) it is possible to meet the deadlines required for triggering the kickers.

As an additional information, the b2b system records some values of signal propagation which is important to diagnose malfunctions of the b2b system and potential violations of upper bound latency:

• t0I-CBS: time difference of the phase measurement (injection) to B2BS. This is in the order of 600 us.
• KTE-fin and fin-CBS: in this mode, the calculations by the CBU consume a significant amount of time. Thus, the 'the end' of CBU activity is shifted much closer towards the deadlines for triggering the kickers.

This mode requires the following messages.

• CMD_B2B_START (B2BS)
• CMD_B2B_PMEXT (PME)
• CMD_B2B_PREXT (PRE)
• CMD_B2B_PMINJ (PMI)
• CMD_B2B_PRINJ (PRI)
• CMD_B2B_TRIGGEREXT (KTE)
• CMD_B2B_DIAGKICKEXT (KDE)
• CMD_B2B_TRIGGERINJ (KTI)
• CMD_B2B_DIAGKICKINJ (KDI)
• CMD_B2B_DIAGEXT (PDE)
• CMD_B2B_DIAGINJ (PDI)

Further Reading

Kicker Timing

More details on kicker timing are described here.

Triggering MIL Based and Legacy Equipment

This is described here.

Timing of FESA Classes

The timing of FESA classes relevant for the b2b system is described here.

-- DietrichBeck - 9 December 2021