General Machine Timing System at GSI and FAIR
The FAIR facility involves a long chain of accelerators which need to be tightly synchronized. This is achieved by the General Machine Timing (GMT) system, a distributed event generation system based on the notion of time. Time synchronization is achieved by using White Rabbit (WR)
, a fully deterministic Ethernet-based field bus for clock transfer and synchronization. The key components of the GMT are a so-called Data Master (DM) that schedules actions by broadcasting messages, a WR network and Timing Receiver (TR) nodes executing machine relevant actions on time.
The primary tasks of the timing system are the following.
- Time-Synchronization of ~2000 - 3000 nodes with sub-ns accuracy over fiber lengths of up to 2 km.
- Distribution of TAI counters with ns accuracy.
- Generation of timing events for synchronization of equipment.
The aim of this timing web is to provide an overview on the timing system and its relationship to other systems. Moreover, it is the central point for documentation. Please note, that the timing system is still under development.
Scheduled downtime of the GMT production system as agreed with operations
are marked with bold
Shutdown works by other systems may affect timing system operation and are listed as 'informative' (please don't blame us!).
- 24-25 January 2022, maintenance works, timing system not available
Schedule for the next weeks based on shutdown planning
. For an overview, click here
- 22 December 2021
- more tests with 'booster mode'
- -> good transfer timing from UNILAC to SIS18 can be achieved with an efficiency exceeding 98% (click)
- 16 December 2021
- maintenance window closed
- 15 December 2021
- 03 December 2021
- dry-run ends
- we had two weeks of smooth GMT operation
- 23 November 2021
- for the first time, the so-called
booster mode has been demonstrated (dry-beam)
- dry-beam is injected, accelerated and extracted from SIS18 with a repetition rate of 2.7 Hz
- the key challenge is tight synchronization of SIS18 and the 50 Hz machine UNILAC at this high rate
- 22 November 2021
- dry-run starts
- new handshake mechanism Data Master <-> UNILAC Pulszentrale successfully established
- 05 November 2021
- maintenance window closed
- 03 November 2021
- 01 November 2021
- maintenance window starts (full week)
- 21-September 2021
- uhps, the Grandmaster had the wrong time
- 15-September 2021
- electricity is back again
- central equipment of the White Rabbit network is back; the Data Masters will be supplied with new schedules soon
- some White Rabbit switches have been switched-off by other groups
- NB: there is ongoing work on the cooling water system and we have only limited cooling power available
- 10-September 2021, central equipment of the White Rabbit networks has been switched off to prevent damage during the upcoming power-cut
- 17-Jul-2021: the 2021 beam time ends today; we had almost 6 months of successful beam operation. We had ...
- very smooth operation with only very few on-call services
- a few failures of the Data Master; this issue is understood and we had no failures in the past two months
- a few failures in the White Rabbit network; this issue is understood and has already been fixed (click)
- a few failures due to unexpected power-cuts
- 16-Jun-2021: some White Rabbit switches lost synchronisation; fixed
- 12-May-2021: Data Master restarted
- first beam transfer between ring machines using White Rabbit based kicker timing
- 'bunch-to-coasting-beam' SIS18 -> ESR
Screenshot by our colleagues from Ring RF. Bunches from the synchrotron SIS18 (magenta) are transferred into the storage ring ESR (orange)
- 03-Mai-2021: Epoch of White Rabbit PTP is back to correct value (UTC-offset 37s)
- 27-Apr-2021: many White Rabbit Switches needed a restart; White Rabbit time is off by one second
- 21-Apr-2021: Data Master restarted
- 19-Apr-2021: restart of a few White Rabbit switches after a power-cut in building BG 1
- 08-Apr-2021: restart of a few White Rabbit switches after a power-cut in building BG 1
- 30-Mar-2021: Data Master restarted
- 09-Mar-2021: White Rabbit grandmaster lost synchronization to GPSDO; reboot of all WRS
- 04-Mar-2021: beam in CRYRING; beam operation in the full facility
- 02-Mar-2021: Data Master restarted
- 28-Feb-2021: in February ...
- ... we had regular operation with SIS18 and ESR and a multitude of experiments
- ... no relevant issues with timing system
- ... but we participate in a bug hunt related to ramped devices
- 07-Feb-2021: beam extracted from SIS18!
Screenshot of equipment by the Beam Instrumentation group in the main control room. green: bunches in SIS18, yellow: differential probe signal from kicker magnet, red: four bunches extracted from SIS and on the way to HHD. The signals pass cables of different length and are not aligned to each other.
- 01-Feb-2021: commissioning of facility starts, on-call service
- 19-Jan-2021: maintenance timing system ends
- 18-Jan-2021: maintenance timing system starts
- updated all WRS to new firmware
- 15-Jan-2021: dry-run ends
- 11-Jan-2021: dry-run starts
- 7 February 2021