Feather M4 with FreeRTOS and F': not starting after adding a single custom Component

[AR-Timm]

Hello I'm trying to run F' on an Adafruit Feather M4 CAN Express following the reference project (https://github.com/fprime-community/fprime-featherm4-freertos-reference). Setting up and running the ReferenceDeployment in the Repository works without any errors, but when I try to add a Component the Software won't start. The Code still compiles without any errors.

To test this I added a basic Component which just adds two numbers and returns them.

Test.cpp:

void Test ::Basic_Test_cmdHandler(FwOpcodeType opCode, U32 cmdSeq, U32 Num_1, U32 Num_2) {
    U32 Result = Num_1 + Num_2;
    this->log_ACTIVITY_HI_Return_Number(Result);
    this->cmdResponse_out(opCode, cmdSeq, Fw::CmdResponse::OK);
}

  void Test ::schedIn_handler(FwIndexType portNum, U32 context) {
    FwSizeType numMsgs = this->m_queue.getMessagesAvailable();
    for (FwSizeType i = 0; i < numMsgs; ++i) {
        (void) this->doDispatch();
    }
  }

Test.fpp:

module trying {
    @ test
    queued component Test {

        async command Basic_Test(
            Num_1: U32
            Num_2 :U32
        ) opcode 0

        @ Example port: receiving calls from the rate group
        sync input port schedIn: Svc.Sched

       @ Example event
        event Return_Number(example: U32)\
            severity activity high\
            id 0\
            format "Number is {}"

        @ Port for requesting the current time
        time get port timeCaller

        @ Port for sending command registrations
        command reg port cmdRegOut

        @ Port for receiving commands
        command recv port cmdIn

        @ Port for sending command responses
        command resp port cmdResponseOut

        @ Port for sending textual representation of events
        text event port logTextOut

        @ Port for sending events to downlink
        event port logOut

        @ Port for sending telemetry channels to downlink
        telemetry port tlmOut 
    }
}                   

Test.hpp:

private:
    void Basic_Test_cmdHandler(FwOpcodeType opCode,  //!< The opcode
                               U32 cmdSeq,           //!< The command sequence number
                               U32 Num_1,
                               U32 Num_2) override;

    void schedIn_handler(
        FwIndexType portNum, //!< The port number
        U32 context //!< The call order
    ) override;                           

I tried adding it as an active and also as an queued component and it didn't work for either one.
instances.fpp:

instance Test: trying.Test base id 0x0500 \
    queue size Default.QUEUE_SIZE\
    # Comment for queued/ UnComment for active
    # stack size Default.STACK_SIZE \
    # priority 103

topology.fpp:

instance Test
    connections RateGroups {
      # Block driver
      rateDriver.CycleOut -> rateGroupDriver.CycleIn

      # Rate group 1
      rateGroupDriver.CycleOut[Ports_RateGroups.rateGroup1] -> rateGroup1.CycleIn
      rateGroup1.RateGroupMemberOut[0] -> tlmSend.Run
      rateGroup1.RateGroupMemberOut[1] -> comDriver.schedIn
      rateGroup1.RateGroupMemberOut[2] -> Test.schedIn
    }

I scanned the UART-port using minicom on the connected computer and got those messages after which nothing more was transmitted.

EVENT: (256) (2:0,510564) DIAGNOSTIC: OpCodeRegistered : Opcode 0x500 registered to port 0 slot 0                    
                                                                                                 EVENT: (256) (2:0,510790) DIAGNOSTIC: OpCodeRegistered : Opcode 0x100 registered to port 1 slot 1
                                                                                                                                                                                                 EVENT: (22
                                                                                                                                                                                                          3
                                                                                                                                                                                                          4
                                                                                                                                                                                                          5
                                                                                                                                                                                                          6
                                                                                                                                                                                                          7
                                                                                                                                                                                                          8
                                                                                                                                                                                                          e
Program Started

This is with the queued Component, with the active one the "Program Started" line will not be printed.

for reference the
Main.cpp:

void setup() {
    // Initialize OSAL
    Os::init();
	
    // Setup Serial and Logging
    Serial.begin(115200);
    while (!Serial) ;
    static_cast<Os::Arduino::StreamConsoleHandle*>(Os::Console::getSingleton().getHandle())->setStreamHandler(Serial);
    
    // Object for communicating state to the reference topology
    ReferenceDeployment::TopologyState inputs;
    inputs.uartNumber = 0;
    inputs.uartBaud = 115200;
    
    // Setup topology
    ReferenceDeployment::setupTopology(inputs);
    xTaskCreate(RateLoop, "RateLoop", 1600, NULL, 1, NULL);

    Fw::Logger::log("Program Started\n");
    vTaskStartScheduler();
}

I added a few Serial.println("..."); Commands in the main.cpp file of the ReferenceDeployment
when my Test-Component is a queued Component it gets to
vTaskStartScheduler();
which is the last line of the setup codeblock. The same happens in an unmodified ReferenceDeployment.

when the component is an active one it stops during this line:
ReferenceDeployment::setupTopology(inputs);

1. Is there an Error with how I have set up my Test component ?
2. I am using the CAN variant of the Feather M4 could that be a problem ?
3. Does anyone have experience with running F' on an Feather M4 and experienced similar problems ?

[LeStarch]

I don't have direct experience with Feather M4 + FreeRTOS, however; on these very tiny systems there are things that you can check. When setupTopology fails, it is likely because of a sizing issue in configuration. I've provided references from fprime-zephyr-reference as that is what I have used.

1. Make sure your project's command dispatcher config is large enough to hold are your commands. Count the commands in your dictionary and make sure they are less than the config.

Here is the configuration from fprime-zephyr-reference: https://github.com/fprime-community/fprime-zephyr-reference/blob/9fa3b0aca65a68f94569119306be4c8509713874/FprimeZephyrReference/project/config/CommandDispatcherImplCfg.hpp#L14. Any additional command in fprime-zephyr-reference requires an increase in this number

2. Repeat for telemetry channels (TlmChan) or packets (TlmPacketizer). Here is the config for tlm packetizer in zephr. https://github.com/fprime-community/fprime-zephyr-reference/blob/devel/FprimeZephyrReference/project/config/TlmPacketizerCfg.hpp

The other major issue could be out-of-memory problems. Make sure to configure queue sizes to what you need, rather than just some large value....as queues put pressure on memory. Same with stack sizes.

@laboratory10 might have more insight.

[AR-Timm]

Thank you for your fast Reply.
It was indeed a out-of-memory problem.
Also I didn't know that you had to update the command dispatcher & Telemetry configs. It was not the error, but it's good to know for when i run into that problem.

[laboratory10]

This sounds a lot like memory issues. Take a careful look at the sizing of the FreeRTOS heap and how precisely it is filling up. The best way to do this is by printing how much memory remains available while the topology is being setup. I remember when I initially had issues like this I was printing it after each task was initialized in the autocoded startTasks function.

Something else that might help is if you look at some of my more recent but not yet merged branches. OsResources for example contains a component I made specifically to better understand how memory was being used. Once I made that component, I made several changes to heap size, queue sizes, and various other configuration changes to free up memory for more components.

[laboratory10]

I've spent some time thinking about this and I'm not sure I'll be able to do much more without the board. Are you still stuck? I might purchase it.

Even though I don't have the board, I did make the changes you made to the FQBN and ran fprime-build with the -v (verbose) option. One thing I noticed when I did is that it was still pulling in the variant.cpp file from the feather_m4 variant directory and not the feather_m4_can one. In order to get it to pull in the correct one, I had to do a purge and fully regenerate the build files.

Can you try that? Once it runs, search your build output and try to figure out if variant.cpp is coming from the right directory.

[AR-Timm]

I'm somewhat stuck. for now I decided to try using a MCP2515 for the CAN interface and stay with the feather_m4 variant.

if I run it with -v it says the the variant.cpp is coming from the feather_m4_can directory.
i compared the variant.cpp.d file in the build-fprime-automatic-FeatherM4_CAN_FreeRTOS/arduino-cli-sketch/core/ folder with the one from the arduino blinking example and they are the same.

[laboratory10]

That's too bad. Would it still be useful to you to have the proper variant file in use if I order the board and try to solve it? I'm placing an order for various parts tomorrow and can add this board to my list if having an answer to this in a few weeks would still be of help to you.

[AR-Timm]

it would be of help to me and greatly appreciated.

[laboratory10]

Ok, I have the board now but I am having trouble replicating your issue. When I clone the reference project and change FeatherM4_FreeRTOS.cmake to:

set(ARDUINO_FQBN "adafruit:samd:adafruit_feather_m4_can")
add_compile_options(-D_BOARD_FEATHERM4_CAN -DUSE_BASIC_TIMER)

and compile and flash it to the board everything works. Next, I tried to mess with the canram region of memory like you did. I added this to main.cpp

extern "C"{
    static uint32_t can_message_buffer[2048] __attribute__((section (".canram"), used)) ;
}

and to make sure the region wasn't zeroed out by the compiler (I had trouble getting it to compile with a non-zero canram section) I added this as the first line of the setup() function:

memset(can_message_buffer, '\0', 2048);

When I compile and run all of that everything still works fine. How exactly did you alter your linker script to put the canram region after data and why precisely did you feel compelled to do so? If you revert back to the original linker script and try changes similar to what I have here does it work for you?