White Rabbit Timing in FESA 3.0.0

Steps to create a White Rabbit triggered real-time event

Class Design

The design needs:

• RT Action
• Logical Event of type "timing"
• Scheduling unit referencing the logical-event and the RT-action

Deployment Unit

The DU needs a concurrency layer containing a reference to the scheduling unit in the design.

The executable needs to be of type 'mixed'.

Instantiation File

The instantiation file needs:

Timing configuration at: classes->[class name]->device-instance->configuration

Set timingDomain FAIR, accelerator, accelerator, zone, mainMuxCriterion to multiplex by sequence or by beam process.

Event configuration at: classes->[class name]->events-mapping->[event name]->event-configuration->Timing->hardware-event

Set the event configuration name and event code (e.g.EVT_START_RF#1)

Event mapping at: classes->[class name]->device-instance->event-mapping->[event name]->event-configuration-ref

Select the event configuration defined above.

Event Behaviour

The FESA framework will register with the timing network to receive events that match the accelerator zone and event numbers given. The framework will always receive BeamProcess Start and Sequence Start events to track timestamps, but will not raise an RT-Action unless specified in the instantiation file.

Multiplexing Context

The timing event triggers the creation of a multiplexing context that contains the event timing details that is passed to the RT-Action.

    uint32_t formatId;
    uint32_t groupId;
    uint32_t eventNumber;
    uint32_t sequenceId;
    uint32_t sequenceCounter;
    uint32_t beamProcessId;
    uint32_t beamProductionChainId;

    uint64_t eventExecutionTimestamp;
    uint64_t sequenceTimestamp;
    uint64_t beamProcessTimestamp;
    uint64_t beamProductionChainTimestamp;
    uint64_t acquisitionTimestamp;

Access to SAFTlib functions

White Rabbit functions are available in the TimingEventSourceWR class

Static Access

#include <fesa-core-gsi/RealTime/TimingEventSourceWR.h>
...
Device* device = WRTimingTestServiceLocator_->getDeviceCollection()[0];
...
fesaGSI::TimingEventSourceWR::registerTrigger(device, eventNo, offset, tag);

Instance Access

#include <fesa-core-gsi/RealTime/TimingEventSourceWR.h>
#include <fesa-core/RealTime/AbstractEventSourceFactory.h>
... 
std::string className = this->WRTimingTestServiceLocator_->getClassName(); // substitute your FESAClassName for WRTimingTest
fesaGSI::TimingEventSourceWR* evtSource = dynamic_cast<fesaGSI::TimingEventSourceWR*>(fesa::AbstractEventSourceFactory::getEventSource(className, "Timing")); 
...
evtSource->registerTrigger( device, eventNo, offset, tag );

SCU bus tags

To generate tags on the SCU bus in response to a timing event:

void TimingEventSourceWR::registerTrigger( AbstractDevice* device, int eventNo, uint64_t offset, uint32_t tag );

The groupID is retrieved from the instantiation file via the device pointer.

Where offset is the time delay in ns after event before tag is written

To stop an event responding:

void TimingEventSourceWR::deregisterTrigger( AbstractDevice* device, int eventNo );

To manually generate a tag on the SCU bus:

void TimingEventSourceWR::injectTag( uint32_t tag);

SCU Environment

The SCU must be running the dbus and saftlib daemons. Example ps output:

 1611 dbus       0:34 /bin/dbus-daemon --system
 1623 root       2:47 /sbin/saftd baseboard:dev/wbm0

https://www-acc.gsi.de/wiki/bin/view/Timing/TimingSystemHowBuildingDeployment

Use the timing tool dm-snoop to check White Rabbit events are reaching the SCU.

How to configure an SCU

To run the dbus and saftlib daemons automatically on an SCU after reboot create a link to the saftlib initialization script in the nfsinit folder for the SCU:

matthies@asl733:scuxl0052>pwd
/common/export/nfsinit/scuxl0052
matthies@asl733:scuxl0052>ll
lrwxrwxrwx 1 matthies bel  19 Jul  9 13:45 20_etherbone -> ../global/etherbone
lrwxrwxrwx 1 matthies bel  25 Jul  9 12:03 30_saftlib -> ../global/saftlib_fesa232
lrwxrwxrwx 1 root     root 20 May 12 11:24 50_fesa -> ../global/fesa_64bit

The scripts will be run in ascending order indicated by the numbers at the beginning.

The etherbone script is used to load tools such as eca-ctl in the RAMdisc of the SCU. The saftlib script saftlib_fesa232 in /common/export/nfsinit/global/ will load the saftlib version used by FESA3 2.3.2 and its required dependencies into the RAMdisc of the SCU and it will replace the etherbone library to match the used saftlib version. The FESA script will set up the SCU for a 64 bit environment expected by 64 bit FESA software.

Multiple instances per SCU

The SAFTlib daemon controls access to the timing hardware. FESA classes create proxy objects to interact with the SAFTlib daemon. This allows multiple FESA classes to execute on the same SCU and register for the same timing event. The number of events that can be listened to is limited by the condition table of the ECA unit.

Accessing SAFTlib directly

Using the Saftlib Directory a FESA class can connect to the Saftlib Daemon without going through the FESA framework. Saftlib tracks ownership of events and actions so events registered with the FESA framework may be seen but not modified. A class may have its own DBUS event loop and set of events instead of - or in addition to - a timing event source registered by the framework.

As an example, a skeleton custom event source for a SoftwareAction.

Create a Custom Event Source, link to RTAction, Set EventMapping.

Required Headers:

#include <giomm.h>
#include <SAFTd.h>
#include <TimingReceiver.h>
#include <SoftwareActionSink.h>
#include <SoftwareCondition.h>

Class members:

Glib::RefPtr<saftlib::TimingReceiver_Proxy> timingReceiver_;
Glib::RefPtr<saftlib::SoftwareActionSink_Proxy> timingActionSink_;
Glib::RefPtr<Glib::MainLoop> timingLoop_;
std::vector<Glib::RefPtr<SoftwareCondition_Proxy> > conditions_;

Constructor / Init

       timingLoop_ = Glib::MainLoop::create();
       std::map<Glib::ustring, Glib::ustring> timingDevices = saftlib::SAFTd_Proxy::create()->getDevices();
       timingReceiver_ = saftlib::TimingReceiver_Proxy::create(timingDevices["baseboard"]);
       timingActionSink_ = saftlib::SoftwareActionSink_Proxy::create(timingReceiver_->NewSoftwareActionSink(""));


void CustomTiming::connect(const boost::shared_ptr<fesa::EventElement>& eventElement)
{
    guint64 eventID =   0x0fa0001000000000;
    guint64 eventMask = 0xfffffff000000000;
    gint64 offset = 0;
    try
    {
       Glib::RefPtr<SoftwareCondition_Proxy> condition = SoftwareCondition_Proxy::create(timingActionSink_->NewCondition(true, eventID, eventMask, offset, 0));
       condition->Action.connect(sigc::mem_fun(this, &CustomTiming::on_action)); // register object method as callback function
       conditions.push_back(condition); // the smart pointer must stay alive for actions to occur
    }
    catch (const Glib::Error& error)
    {
    }
}

void CustomTiming::wait(boost::shared_ptr<fesa::RTEvent>& eventToFire)
{
    timingLoop_->run(); // this blocks until an action occurs
    createEvent(eventToFire, CustomTiming::defaultEvent);
}

// callback function
void CustomTiming::on_action(guint64 id, guint64 param, guint64 time, guint64 overtime,  bool late, bool delayed, bool conflict)
{
    // do something with timing information
    timingLoop_->quit();
}

More Information

https://www-acc.gsi.de/wiki/pub/Timing/TimingSystemDocumentsSaftlib/saftlib.pdf

https://www-acc.gsi.de/wiki/Timing/TimingSystemEventNumbers

https://www-acc.gsi.de/wiki/Timing/TimingSystemGroupsAndMachines

SAFTlib Version History

FESA uses snapshots of SAFTlib pending a versioning system.

07.07.2015

First Release

18.08.2015

Fixes:

Event triggers for SCUbus Tags with adjacent EventIDs are accepted

Action offsets greater than 2^32ns work correctly.

Negative offsets for actions work correctly.

No Late Errors for events that have not been subscribed to

High 32-bit of payload is correct, no longer copied from eventid.

Changes:

saft-ctl tool replaces the functionality of dm-snoop and eca-ctl

Interface:

Fixed inconsistent signedness of guard and offset parameters.

-- DominicDay - 06 Jul 2015