STM32H723 + Zephyr + F´: RAM overflow after enabling telemetry (TlmSend)

[rohamkaveie]

Issue Summary

I am using fprime-community/fprime-stm32h7-zephyr-reference repository as a reference to develop my project and when i enable the instance of tlmSend in Topology.fpp, i get RAM overflow error and the build (&flash) of application does not get completed.

Environment

• MCU: STM32H723 (Cortex-M7)
• RTOS: Zephyr
• Framework: F´ (F Prime)

The Problem

After cloning the fprime-stm32h7-zephyr-reference repo, I noticed the TlmSend instance in the topology file was commented out. I assumed this was why telemetry wasn’t working, so I uncommented those lines and wired the ports. As soon as I rebuilt/flashed, the build failed with a RAM overflow during link/flash.

FAILED: zephyr/zephyr_pre0.elf zephyr/zephyr_pre0.map /home/roham/SPARCSMission/SPARCSApp/build-fprime-automatic-zephyr/zephyr/zephyr_pre0.map 
: && ccache /usr/local/zephyr-sdk-0.17.2/arm-zephyr-eabi/bin/arm-zephyr-eabi-g++  -gdwarf-4 zephyr/CMakeFiles/zephyr_pre0.dir/misc/empty_file.c.obj -o zephyr/zephyr_pre0.elf  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  -T  zephyr/linker_zephyr_pre0.cmd  -Wl,-Map=/home/roham/SPARCSMission/SPARCSApp/build-fprime-automatic-zephyr/zephyr/zephyr_pre0.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/libarch__arm__core.a  zephyr/arch/arch/arm/core/cortex_m/libarch__arm__core__cortex_m.a  zephyr/arch/arch/arm/core/mpu/libarch__arm__core__mpu.a  zephyr/lib/libc/newlib/liblib__libc__newlib.a  zephyr/lib/libc/common/liblib__libc__common.a  zephyr/drivers/interrupt_controller/libdrivers__interrupt_controller.a  zephyr/drivers/clock_control/libdrivers__clock_control.a  zephyr/drivers/console/libdrivers__console.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/pinctrl/libdrivers__pinctrl.a  zephyr/drivers/reset/libdrivers__reset.a  zephyr/drivers/serial/libdrivers__serial.a  zephyr/drivers/timer/libdrivers__timer.a  modules/hal_stm32/stm32cube/lib..__modules__hal__stm32__stm32cube.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  -L/home/roham/SPARCSMission/SPARCSApp/build-fprime-automatic-zephyr/zephyr  lib/zephyr/libOs_File_Stub_Implementation.a  lib/zephyr/libOs_Generic_PriorityQueue_Implementation.a  lib/zephyr/libOs_Generic_Types.a  lib/zephyr/libOs_Cpu_Stub_Implementation.a  lib/zephyr/libOs_Memory_Stub_Implementation.a  lib/zephyr/libfprime-zephyr-deployment.a  lib/zephyr/libReferenceDeployment_Top.a  lib/zephyr/libComponents_Led.a  lib/zephyr/libfprime-zephyr_Drv_ZephyrGpioDriver.a  lib/zephyr/libfprime-zephyr_Drv_ZephyrRateDriver.a  zephyr/kernel/libkernel.a  lib/zephyr/libfprime-zephyr_Drv_ZephyrUartDriver.a  lib/zephyr/libDrv_Ports.a  lib/zephyr/libfprime-zephyr_Svc_FatalHandler.a  lib/zephyr/libSvc_ActiveLogger.a  lib/zephyr/libSvc_Fatal.a  lib/zephyr/libSvc_ActiveRateGroup.a  lib/zephyr/libSvc_AssertFatalAdapter.a  lib/zephyr/libSvc_BufferManager.a  lib/zephyr/libSvc_ChronoTime.a  lib/zephyr/libSvc_CmdDispatcher.a  lib/zephyr/libSvc_CmdSequencer.a  lib/zephyr/libSvc_Seq.a  lib/zephyr/libSvc_ComQueue.a  lib/zephyr/libSvc_ComStub.a  lib/zephyr/libDrv_ByteStreamDriverModel.a  lib/zephyr/libSvc_FileDownlink.a  lib/zephyr/libSvc_FileDownlinkPorts.a  lib/zephyr/libSvc_FileManager.a  lib/zephyr/libSvc_FileUplink.a  lib/zephyr/libFw_FilePacket.a  lib/zephyr/libCFDP_Checksum.a  lib/zephyr/libSvc_FprimeDeframer.a  lib/zephyr/libSvc_FprimeFramer.a  lib/zephyr/libSvc_FprimeRouter.a  lib/zephyr/libSvc_FrameAccumulator.a  lib/zephyr/libFw_Ports_SuccessCondition.a  lib/zephyr/libSvc_Ports_CommsPorts.a  lib/zephyr/libUtils_Types.a  lib/zephyr/libSvc_FprimeProtocol.a  lib/zephyr/libSvc_Ccsds_Types.a  lib/zephyr/libSvc_Health.a  lib/zephyr/libSvc_WatchDog.a  lib/zephyr/libSvc_PassiveConsoleTextLogger.a  lib/zephyr/libSvc_PrmDb.a  lib/zephyr/libSvc_RateGroupDriver.a  lib/zephyr/libSvc_Cycle.a  lib/zephyr/libSvc_SystemResources.a  lib/zephyr/libSvc_TlmChan.a  lib/zephyr/libSvc_Sched.a  lib/zephyr/libFw_CompQueued.a  lib/zephyr/libFw_Comp.a  lib/zephyr/libOs.a  lib/zephyr/libOs_File.a  lib/zephyr/libUtils_Hash.a  lib/zephyr/libOs_Cpu.a  lib/zephyr/libOs_Memory.a  lib/zephyr/libOs_Console.a  lib/zephyr/libOs_Task.a  lib/zephyr/libOs_Mutex.a  lib/zephyr/libOs_Queue.a  lib/zephyr/libOs_RawTime.a  lib/zephyr/libFw_Buffer.a  lib/zephyr/libSvc_Ping.a  lib/zephyr/libFw_Prm.a  lib/zephyr/libFw_Cmd.a  lib/zephyr/libFw_Log.a  lib/zephyr/libFw_Tlm.a  lib/zephyr/libFw_Fpy.a  lib/zephyr/libFw_Com.a  lib/zephyr/libFw_Time.a  lib/zephyr/libFw_Port.a  lib/zephyr/libFw_Obj.a  lib/zephyr/libFw_Logger.a  lib/zephyr/libFw_Types.a  lib/zephyr/libdefault_config.a  zephyr/arch/common/libisr_tables.a  -mcpu=cortex-m7  -mthumb  -mabi=aapcs  -mfpu=fpv5-d16  -mfloat-abi=hard  -mfp16-format=ieee  -fuse-ld=bfd  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn  -Wl,-no-pie  -L"/usr/local/zephyr-sdk-0.17.2/arm-zephyr-eabi/arm-zephyr-eabi"/lib/thumb/v7e-m+dp/hard -L"/usr/local/zephyr-sdk-0.17.2/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/thumb/v7e-m+dp/hard" -lstdc++ -lm -lc -lgcc -lc && cd /home/roham/SPARCSMission/SPARCSApp/build-fprime-automatic-zephyr/zephyr && /home/roham/SPARCSMission/SPARCSApp/fprime-venv/lib/python3.12/site-packages/cmake/data/bin/cmake -E true
  __ASSERT() statements are globally ENABLED
/usr/local/zephyr-sdk-0.17.2/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: zephyr/zephyr_pre0.elf section `noinit' will not fit in region `RAM'

/usr/local/zephyr-sdk-0.17.2/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: region `RAM' overflowed by 28288 bytes

What I changed

You can see following changes i did before getting that error. I just uncomment these lines in topology.fpp (they are not together like this i just copied them so you can find them in the file):

instance tlmSend 
telemetry connections instance tlmSend
tlmSend.PktSend             -> comQueue.comPacketQueueIn[Ports_ComPacketQueue.TELEMETRY]
rateGroup1.RateGroupMemberOut[1] -> tlmSend.Run

Certainly I uncomment these lines in instances.fpp too:

  instance tlmSend: Svc.TlmChan base id 0x0C00 \
    queue size Default.QUEUE_SIZE \
    stack size Default.STACK_SIZE \
    priority 97

What I've Tried to Fix This

• Change the size of Heap and Stack memory for the instances in instances.fpp. result: the amount of overflowed memory does not change even a byte.
• Change the prj.conf to disable unnecessary configuration for zephyr. result: the amount of overflowed memory decreased only 3KB (28KB ahead)

Questions

1. How can i solve this problem?
2. Is this a correct way to enable Telemetry?
3. Should i probably focused on zephyr footprint or this issue is related to Fprime framework?
4. Any advice on where to start to solve this problem?

Any help would be hugely appreciated!
Thank you for your kind attention.

[kevin-f-ortega]

Hi. Find below answers to your questions

1. You need to decrease how much memory your FSW uses:

   • If you are not using files in your project, then remove all file-related components from your instance.fpp and topology files.
   • Use PassiveRateGroup instead of the ActiveRateGroup.
   • Finer tune this file https://github.com/fprime-community/fprime-stm32h7-zephyr-reference/blob/main/config/TlmChanImplCfg.hpp
   • Instead of Svc.TlmChan, you can use https://github.com/fprime-community/fprime-baremetal/tree/main/fprime-baremetal/Svc/TlmLinearChan. That would require you to add the fprime-baremetal repo to your lib directory.
   • Override this config file https://github.com/nasa/fprime/blob/0a9c0570137c95ebd01cdc42ca3a1b5b12d975d4/default/config/AcConstants.fpp (you will have to copy it over to https://github.com/fprime-community/fprime-stm32h7-zephyr-reference/tree/main/config and update the CMakelists.txt file). Do some fine tuning to decrease the number of ports. Use the minimum number of ports needed for each port.

2. Yes, that is correct.

3. You can do both, but I suspect you'll get more progress by doing the steps I mentioned above.

4. What you tried is a good attempt, but for the issue you are seeing, you need to decrease the .text size of your software. That is done by the steps mentioned in answer 1.

[rohamkaveie]

Hello again.
Thank you for your answer but the issue is not solved yet.
In order of things you said, i did following steps:

1. To launch tlmSend, i created a new instance from PassiveRateGroup Component (alongside other ActiveRateGroup instances) but it did not help. In facts i think the issue is not due to the connections at all, because when i comment out all lines related to connections and just leave the line "instance tlmSend" in topology.fpp file, the memory RAM gets still overflowed.
2. Tuning TlmChanImplCfg.hpp file does not change the value of overflow in any ways.
3. I actually tried to use TlmLinearChan from baremetal repository but i could not make it compatible for fprime-zephyr repository
4. Doing the last step does not help too (The value of memory overflow does not change again).