Memory management unit (MMU) on SCU (DDR3 on SCU3 and SRAM on SCU4)

Table of Contents

Organization of memory sections

The SCU has a relatively large data memory. In the SCU 3, this memory is implemented as DDR3 RAM with a maximum size of 128 MB, and in the future the SCU 4 will have an SRAM memory with a size of 32 MB.
It is clear that several independent applications (LM32 and Linux) use this memory.
For this reason it becomes necessary to implement a central memory-administration respectively a memory management unit (MMU) which allocates for each application (LM32 or linux) using this memory one or more memory segments.
The MMU shall be as easy as possible. This will be accomplished by using of a single linked list.
The smallest addressable unit of the DDR3 RAM on SCU 3 is 64 bit therefore - in the future - the smallest addressable unit of the SRAM on SCU4 will also be 64 bits for compatibility reasons.

The documentation about the DDR3 VHDL-macro written by Karl Heinz Kaiser in german can be found here.

The following figure demonstrates the SCU-data memory with four segments (payload 1 to 4) which can be used independently from different applications.

Start descriptor

As shown above the memory allocating list starts with a so called "start descriptor" usefully at address 0x00000000.

• The first 32-bit (red) field of the start descriptor includes a so called magic number to indicate the begin of the linked list and whether the list is already created or not.
  The value of this magic number is 0xAAFF0055
• The second 32-bit field (green) includes the start index, respectively the address offset in 64 bit units, of the first payload descriptor. Has this field the value zero, so is the list yet empty.
• Obtaining the same size like the payload descriptors (16 byte) the next 64- bit field (gray) is empty respectively reserved for future use.

Payload descriptor

Each reserved memory area for the applications payload has a so called "payload descriptor".

• The first 16 bit field (red) includes the unique tag which identifies to which application(s) the corresponding memory sector belongs.
• The second 16 bit field (yellow) includes a flag field which is still unused at the moment respectively reserved for the future.
• The third 32 bit field (green) includes the start index, respectively the address offset in 64 bit units, which points to the next payload descriptor. Has this field (green) a value of zero, then is this payload descriptor the last descriptor, that means the end of the linked list has reached.
• The fourth 32 bit field (orange) includes the start address, respectively the address offset in 64 bit units, which points to the payload area belonging to this descriptor.
• The fifth 32 bit field (blue) includes the size of the payload area belonging to this descriptor in 64 bit units.

As we can see, it is not absolutely necessary that the payload area belonging to the descriptor directly borders to the descriptor, as shown in the figure, but it is the easiest way to implement it.

Reserved memory tags

At the moment the following menory tags (red field) are already reserved. Defined in header scu_mmu_tag.h.

Name	Value	Meaning
-	0xAAFF	Part of the magic number do not use!
TAG_ADDAC_DAQ	0xFF01	Shared memory for ADDAC-DAQs
TAG_MIL_DAQ	0xFF02	Shared memory for MIL-DAQs
TAG_LM32_LOG	0xFC01	Shared memory for LM32 log messages (lm32-logd)

Sourcecode of the SCU-MMU

CAUTION: It is not mandatory to use the C/C++ source code presented here, but it is absolutely necessary to stick to the convention described above if an application wants to use this memory!

The kernel module for the SCU-MMU is written in C and can be bind to a linux application and to a LM32 application.
The C module is implemented in in the source file scu_mmu.c and its interface in the header file scu_mmu.h.
NOTE: The different byte order between a Linux application (little endian) and an LM32 application (big endian) is of course taken into account.

Furthermore, a C++ wrapper class is available for the frontend group, which usually uses the C++ class "EtherboneConnection" for Etherbone accesses: scu_mmu_fe.cpp and scu_mmu_fe.hpp.
But in this case the source files scu_memory.hpp, scu_ddr3_access.cpp and scu_ddr3_access.hpp, scu_etherbone.cpp and scu_etherbone.hpp are also necessary. And in the future for the SCU4: scu_sram_access.cpp and scu_sram_access.hpp.

The prototype of memory allocator is defined in scu_mmu.h:

MMU_STATUS_T mmuAlloc( const MMU_TAG_T tag, MMU_ADDR_T* pStartAddr, size_t* pLen, const bool create );

• The first parameter is the above discussed tag for identifying the allocated or to allocate memory segment.
• The second parameter points to the start index (in 64 bit units) of the successful created or linked memory segment after the function call.
• The third parameter is a pointer to the desired payload size in 64 bit units. In the case the the memory segment was linked only and not created, then this pointer includes the actual size of the already crated memory segment after this function call.
• The last parameter is a flag which determines whether the memory segment shall be created or only a link to a already present segment shall made.

Returncode:

Name	Value	Meaning
OK	0	Action was successful.
MEM_NOT_PRESENT	-1	Wishbone device of RAM not found.
LIST_NOT_FOUND	-2	Start of linked list resp. magic number not found. This is only the case when only a link to an existing segment will made, otherwise the start descriptor becomes created as well.
TAG_NOT_FOUND	-3	Segment with the given tag not found. This is only the case when only a link to an existing segment will made.
ALREADY_PRESENT	-4	Memory segment with the given tag had already created, that means the in the third parameter desired size becomes overwritten by the actual size of the found segment.
OUT_OF_MEM	-5	Requested memory block doesn't fit in physical memory.

Helper Tool "mem-mon" (Memory Monitor)

By the help of the application "mem-mon" (memory monitor) it becomes possible to show the table in the SCU-RAM like discussed above.
Furthermore it is possible to create a payload-sector or deleting all partitions.

For examlpe:

mem-mon scuxl4711.acc.gsi.de -v

Makes this output:

If mem-mon called direct on a SCU so the scu-name as parameter is not necessary.

For more information about mem-mon please read the build-in help of mem-mon by typing:

mem-mon -h

The sourcecode of mem-mon can be found here.

-- UlrichBecker - 09 Jan 2024