PMC Testing and Commissioning Guide




+++ Attention: This guide does not check the whole PCI/PMC standard! +++

+++ Attention: This is NOT for FAT +++

Required components for each PMC device

  • SFP (green/purple)
  • LC cable and a white rabbit switch (Recommended: RUNNING AT LEAST VERSION 4.1, if you can afford this)
  • Quartus (Version 16.0.0 Build 211 was used for this guide)
  • USB cable (micro 2.0)
  • Power supply (if you don't use a crate)
  • Xilinx programmer (Platform Cable USB II, DLC10)
  • Altera USD blaster + PROMO11 adapter
  • 5x LEMO cable(s)
  • A second Timing Receiver with 5 LVTTL IOs (AMC, PMC, Exploder5, ...)
  • $dev is a placeholder for dev/ttyUSB[X] AND dev/wbm[X] - you have to test both interfaces!
  • $saftlib-dev is a placeholder for tr[X], baseboard[X], ...

Additional Resources

Preparation

  1. Check out bel_projects
  2. $ git checkout https://github.com/GSI-CS-CO/bel_projects.git
  3. $ cd bel_projects
  4. $ git checkout 1879cee [https://github.com/GSI-CS-CO/bel_projects/pull/101/commits/1879cee543c0b4a88656b7a11fa8be605787546d]
  5. $ ./fix-git.sh
  6. $ ./install-hdlmake.sh
  7. $ make
  8. (additional) $ make driver-install
  9. (additional) $ make etherbone-install
  10. (additional) $ make saftlib-install
  11. (additional) Use insmod to load the drivers oder restart

Important Steps

Programming the CPLD

  1. Turn on power
  2. Use Cosylab's boxed header to micro USB adapter and connect via JTAG (xilinx programmer)
  3. Run ISE
  4. Load project bel_projects/syn/gsi_pmc/cpld/pmc_prog.xise
  5. Process Menu => Implement Top Module
  6. Tools Menu => Impact
  7. Double-click Boundary Scan
  8. Control-I => pmc_prog.jed
  9. Operations Menu => Program
  10. Turn power off

Programming the FPGA

  1. Turn power on
  2. Connect JTAG via PROMO11 adapter
  3. Open Quartus and program the FPGA (or use the command line: quartus_pgm -c 1 -m jtag -o 'p;pmc.sof')
  4. (optional) Write this bit-stream into the SPI flash: eb-flash $dev pmc.rpd (in case USB and flash chip are already programmed)

Programming the USB Chip

  1. Run 'make' in bel_projects/ip_cores/etherbone-core/hdl/eb_usb_core
  2. Make sure, that no other timing receiver is attached by USB
  3. Erase the USB controller (as root): ./flash-fx2lp.sh -E
  4. Program the USB controller (as root): ./flash-fx2lp.sh
  5. Turn power off and on

Configuring (and programming) the SPI Flash Chip

  1. Program the FPGA again: quartus_pgm -c 1 -m jtag -o 'p;pmc.sof'
  2. Configure the SPI flash chip: eb-config-nv $dev 10 4
  3. [at room temperature] Write the bit-stream into the SPI flash: eb-flash $dev pmc.rpd (in case FPGA is not persistently programmed
  4. Turn power off and on
  5. [at high temperature > crate without cooling] Write the bit-stream into the SPI flash again: eb-flash $dev pmc.rpd
  6. Repeat 5. three times

Formatting the WR EEPROM (MAC address is stored here)

  1. Go to bel_projects/ip_cores/wrpc-sw/tools
  2. Run "make"
  3. ./eb-w1-write $dev 0 320 < sdb-wrpc.bin

Check White Rabbit

  1. eb-console $dev
  2. Type in "gui", white rabbit status should be: locked and calibrated
  3. Remove and apply the fiber cable five times and make sure WR locks again
  4. Press ESC to quit
  5. Power cycle the device and repeat step #3 -> Watch the WR LEDs (the receiver should synchronize again)
Synchronization status should be:
+ Servo state: TRACK_PHASE
+ Phase tracking: ON
You should also see 4 leds at the front panel:
+ red = traffic/no-link
+ blue = link
+ green = timing valid
+ white = PPS
  1. Remove fiber cable
  2. Type in "mode master", node should be able to lock the PLL and become a master
    Quit console
  3. (optional) Test status via Saftlib $ saft-ctl $saftlib-dev -s
Output should look like this:
WR locked, time: 0x152c4f5bbfcf5528
receiver free conditions: 256, max (capacity of HW): 0(256), early threshold: 4294967296 ns, latency: 4096 ns

Check External Reference Clock

  • Provide a 10MHz clock (i.e. from a White Rabbit switch)
  • Drive IO_CLKIN_EN to High $ saft-io-ctl $saftlib-dev -n IO_CLKIN_EN -d 1 (Note: In the foreseeable future it will work like this: $ saft-io-ctl $saftlib-dev -n IO5 -q 1)
  • Open the White Rabbit console again and type in: mode master
  • Check if WR is really locked (LED, eb-console -> GUI, generate clocks, ...)
  • Additionally check if the system works with a 20MHz input clock too
  • (optional) Turn off master mode (mode slave)
  • (optional) Generate a clock on this and on another receiver and compare them

Check EEPROM and set MAC

  1. Run eb-console $dev
  2. (optional) Set MAC address for the device #xy: <<mac setp aa:bb:cc:dd:ee:ff, Control-C
  3. Check given MAC address
  4. Turn power off and on
  5. Run eb-console $dev
  6. Type in "mac", you should see the previously entered MAC address
  7. If you don't have a DHCP server, you can set an ip address by "ip set 192.168.100.xyz"

Check IOs

Front Panel IOs

  1. Get a second uTCA timing receiver (or a receiver with at least 5 LVTTL IOs)
  2. Connect IO1 (device #1) to IO1 (device #2) and so on... [IO1,IO2,IO3,IO4,IO5]
  3. Device #1: Turn output enable for the IOs on $ saft-io-ctl $saftlib-dev -n IO{1,2,3,4,5} -o 1
  4. Device #1: Start a clock on each IO $ saft-clk-gen $saftlib-dev -n IO{1,2,3,4,5} -f 10000000 0 (will generate a 10 MHz clock)
  5. Device #2: Snoop inputs $ saft-io-ctl $saftlib-dev -s
  6. Measure jitter and signal quality with a scope (FTRN specification should be matched)
Slew Rate
  1. Measure the slew rate with a scope (for EVERY IO), should be equal to Exploder5
IO Load Test
  1. Drive every IO with termination
  2. $ saft-io-ctl $saftlib-dev -n IO{1,2,3,4,5,...} -o 1 -t 0/1 -d 1
  3. Measure levels/voltage with a scope
Termination Test
  1. Drive every IO with and without termination
  2. Make sure that the switchable termination works
IO Bandwidth
  1. Generate a 200MHz clock (saft-clk-gen) and measure the clock with a scope
  2. Generate a 125MHz clock (with an additional receiver or reference device) and measure it with the ECA (saft-io-ctl snoop mode)
Output Enable Test
  1. Drive every IO with and without output enable
  2. Make sure you don't see a level change when output enable is turned off
  3. Check the LEDs, they should indicate every activity and the output enable status

Check PCI

  1. Try the following tools:
    • eb-console $dev
    • eb-info $dev
    • eb-ls $dev
  2. Write to internal shared ram and read it back:

    • Get the LM32 shared ram address by eb-ls $dev
    • Example output: 3.2 0000000000000651:81111444 84000 LM32-RAM-Shared
    • Create a dummy file (which will be written into the ram): dd if=/dev/urandom of=foo bs=4k count=1;
    • Write dummy file to the lm32 shared ram: eb-put $dev 0x84000 foo
    • Get the data from the shared ram: eb-get $dev 0x84000/4096 bar
    • Compare both files: cmp foo bar
    • Both files should contain the same data
    • Repeat this test in a loop... (recommended: 10 minutes)

Check PCI interrupts

  1. Create your own schedule (or use the attached one -> schedule.txt)
  2. Start snooping for events: $ saft-ctl $saftlib-dev snoop 0 0 0 -x
  3. Inject events: $ saft-dm $saftlib-dev -fp -n 1 schedule.txt
  4. Verify that you got all event

Check LEDs

  1. $ saft-pps-gen $saftlib-dev -s
  2. Leave the application running and check all LEDs

Check Buttons and Hex Switches

  1. $ saft-io-ctl $saftlib-dev -s
  2. Start rotating switches => Check output of saft-io-ctl
  3. Start pushing buttons => Check output of saft-io-ctl
IO . Edge . Flags .... ID ...................... Timestamp ........... Formatted Date
---------------------------------------------------------------------------------------------------------
PBF Rising .... (0x0) 0xfffe000000000015 0x1532cdaf1fb0beb7 2018-05-28 12:09:35.666667191
PBF Rising .... (0x0) 0xfffe000000000017 0x1532cdaf24a84fe8 2018-05-28 12:09:35.750000616

Check Display

  1. cd bel_projects/tools/display
  2. make
  3. ./simple-display dev/ttyUSBx -s "Hello World!" -d 2

Check debug port HPLA

  1. To be defined

Check OneWire Devices

  1. Go to: tools/commissioning/onewire-scanner
  2. make
  3. run application: ./onewire-scanner dev/ttyUSB0

Output should look like this (OWID should be slightly different):

Scanning for OneWire controller(s) on $dev now...

ID Wishbone Address OWID Serial Code Type
-------------------------------------------------------------------------------
00 0x0000000000060600 -- --- ---
-- --- 00 0x32000009195de328 DS18B20 - Digital Thermometer
-- --- 01 0x89000000ba822c43 DS28EC20 - 20Kb EEPROM

Found 1 OneWire controller(s) on $dev.
Found 2 OneWire device(s)/slave(s) on $dev.

Check OneWire Devices (eb-mon)

  1. Check device with eb-mon
root@pmc_carrier:~# eb-mon dev/wbm0 -w0 -b0 -f0x28
0xbd0000091958c628
root@pmc_carrier:~# eb-mon dev/wbm0 -w0 -t0 -f0x28
45.1875

Additional Tests (done by Cosylab)

MBOX

  1. Put PMC card into MBOX
  2. Perform tests "Check PCI" and "Check PCI interrupts" again

JTAG PMC Settings

  • Check if JTAG <=> PMC bridge works

Address and Data Lines

Write (using PCI) to internal shared ram and read (using USB) it back:
  1. Get the LM32 shared ram address by eb-ls $dev
  2. Example output: 3.2 0000000000000651:81111444 84000 LM32-RAM-Shared
  3. Create a dummy file (which will be written into the ram): dd if=/dev/urandom of=foo bs=4k count=1;
  4. Write dummy file to the lm32 shared ram: eb-put $devPCI 0x84000 foo ($devPCI = dev/wbmX here!)
  5. Get the data from the shared ram: eb-get $devUSB 0x84000/4096 bar ($devUSB = dev/ttyUSBX here!)
  6. Compare both files: cmp foo bar
  7. Both files should contain the same data
  8. Repeat this test in a loop... (recommended: 10 minutes)

This topic: Timing > TimingSystemPMCTestingAndCommissioning
Topic revision: 05 Mar 2019, AlexanderHahn
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback