Concept

Every sensor reading your Python code receives, and every motor command your Python code sends, travels through a five-stage pipeline that spans two processors, two buses, and three IPC mechanisms.

Understanding this pipeline helps you reason about latency (why sensor reads are always slightly stale), reliability (why retained channels exist), and architecture (why from raccoon_transport import Transport is the low-level API and get_transport() is what project services should use).

graph LR
    subgraph STM32["STM32F427 — Hard Real-Time"]
        HW["Physical sensor\nor motor"]
        ISR["ADC1 DMA (250 Hz)\nBEMF ADC2 (800 Hz)\nIMU DMP (50 Hz)\nDigital GPIO"]
        TX["TxBuffer\n(volatile packed struct,\nTRANSFER_VERSION 21)"]
    end

    subgraph SPIBus["SPI2 — Full-Duplex\n20 MHz, ~5 ms poll cycle"]
        WIRE["ioctl(SPI_IOC_MESSAGE)\nTxBuffer ⇄ RxBuffer\nBUFFER_LENGTH_DUPLEX_COMMUNICATION bytes"]
    end

    subgraph Reader["stm32-data-reader (C++, wombat ns)"]
        UNPACK["SpiReal::readSensorData()\nunpack + unit-convert\n(ADC counts → V, EMA filter)"]
        GATE["DataPublisher\nPublishGate 50 Hz cap\n+ L∞ noise epsilon"]
        PUB["LcmBroker\nserialise + dedup\nraccoon::Transport"]
    end

    subgraph SHM["raccoon_ring SHM\n(/dev/shm/raccoon_ring_*)"]
        RING["64-slot × 2 KiB ring\nper channel\nSeqLock, futex wake"]
    end

    subgraph Lib["raccoon-lib (Python)"]
        LCMR["LcmReader\nspinOnce() thread\nfutex_waitv"]
        CACHE["mutex-protected\nper-channel cache"]
        API["motor.get_position()\nanalog.read()\nimu.heading()"]
    end

    HW --> ISR --> TX
    TX --> WIRE --> UNPACK --> GATE --> PUB --> RING
    RING --> LCMR --> CACHE --> API

The reverse direction (Python command → motor output) follows the same path in reverse: Python publishes to a raccoon/motor/N/power_cmd LCM channel, the reader’s CommandSubscriber picks it up, writes the RxBuffer, and the next SPI transfer delivers it to the STM32.

This page traces a complete data path from a physical sensor reading to a value accessed in user Python code, and a complete command path from Python code to motor movement.

Sensor Data Path (STM32 → Python)

Physical sensor
      │
      ▼
STM32 ADC / GPIO / IMU
      │  (interrupt-driven, µs latency)
      ▼
txBuffer (volatile struct in STM32 RAM)
      │  (updated after each measurement cycle)
      ▼
SPI2 DMA transfer to Raspberry Pi
      │  (circular DMA, Pi polls at main-loop rate)
      ▼
stm32-data-reader: SpiReal::readSensorData()
      │  (spi_update() call in C, copies RxBuf → TxBuf,
      │   returns SensorData struct)
      ▼
stm32-data-reader: DataPublisher::publishSensorData()
      │  (serialises fields to LCM messages, publishes
      │   via raccoon::Transport using raccoon_ring SHM)
      ▼
raccoon_ring shared memory (/dev/shm/raccoon_ring_<channel>)
      │  (primary IPC; one ring-buffer file per channel)
      ▼
raccoon-lib: LcmReader (background thread in C++ process)
      │  (raccoon::Transport::spinOnce, updates caches)
      ▼
LcmReader caches (mutex-protected std::unordered_map
      │   / scalar fields)
      ▼
Python binding call (e.g., motor.get_position(),
      analog.read(), imu.heading())

Step 1: STM32 Hardware Layer

Analog sensors and battery: ADC1 runs in continuous circular DMA mode, started once at boot by startContinuousAnalogSampling(). DMA streams results into an oversampling accumulator continuously. TIM6 fires at ANALOG_OUTPUT_INTERVAL (4000 µs = 250 Hz): the accumulator is snapshotted, averaged, VDDA-corrected, and written to txBuffer. There is no separate ANALOG_SENSOR_SAMPLING_INTERVAL constant.

Digital sensors: readDigitalInputs() is called directly from HAL_SPI_TxRxCpltCallback and the result written into txBuffer.digitalSensors. Digital sensors are updated on every single SPI transaction.

BEMF and motor state: TIM6 calls stop_motors_for_bemf_conv() every 1250 µs (one motor at a time, round-robin). After 500 µs settle time, ADC2 DMA conversion runs for the current motor’s two channels. On ADC2 completion, BEMF is processed through the two-stage filter (median-of-3 + IIR α=0.2), offset-corrected, dead-zone applied, and dt-aware position integration runs. The motor control loop re-applies the PWM command. updatingMotorsInSpiBuffer() copies motor_data into txBuffer.motor after confirming SPI2 is idle. Each motor is measured every 4 × 1250 = 5000 µs (200 Hz effective per motor).

IMU: Main loop calls readImu(). When the MPL has new fused data, txBuffer.imu is updated.

Step 2: SPI Transfer

The Pi initiates SPI transactions by calling spi_update() in the C SPI layer. This performs a synchronous ioctl(SPI_IOC_MESSAGE) on /dev/spidev0.0 — a single system call that transfers BUFFER_LENGTH_DUPLEX_COMMUNICATION bytes (the max(sizeof(TxBuffer), sizeof(RxBuffer)) at compile time) in both directions simultaneously.

sequenceDiagram
    participant App as Application main loop
    participant DC as DeviceController
    participant SR as SpiReal
    participant C as C SPI layer (Spi.c)
    participant STM as STM32F427

    Note over App: every 5 ms (mainLoopDelay)
    App->>DC: processUpdate()
    DC->>SR: readSensorData()
    SR->>C: spi_update()
    C->>STM: ioctl(SPI_IOC_MESSAGE)\nPi sends RxBuffer\nSTM32 sends TxBuffer\n(full-duplex, 20 MHz)
    STM-->>C: TxBuffer (sensor data,\nupdateTime, TRANSFER_VERSION 21)
    C-->>SR: return true, rx_buffer populated
    SR->>SR: unpack TxBuffer fields\nbattery: ADC × 3.3 × 11 / 4096\nEMA alpha=0.05
    SR-->>DC: SensorData struct
    DC->>DC: lastSensorData_ = result
    DC-->>App: Result::success()
    App->>App: publishCurrentData()\nif lastUpdate changed

The Pi sends the current RxBuffer (commands: motor modes, targets, servo positions, PID gains, feature flags) while simultaneously receiving the STM32’s TxBuffer (sensor data, motor telemetry, IMU, odometry). Both directions happen in one system call with no gaps.

The protocol version field (TxBuffer.transferVersion = 21, TRANSFER_VERSION 21 in pi_buffer.h) is checked once at startup via spi_probe_version(). A mismatch triggers an automatic firmware reflash on the next update call.

The Pi’s stm32-data-reader calls spi_update() on every main-loop iteration. The default mainLoopDelay is 5 ms (Configuration::mainLoopDelay).

Step 3: stm32-data-reader → LCM

SpiReal::readSensorData() unpacks the raw TxBuffer into a C++ SensorData struct with named fields. Unit conversions that happen here:

• Battery voltage is converted from ADC counts to volts using the known 11× resistor divider and 3.3 V reference, then filtered with a 5% EMA.
• Analog sensor values are passed as raw int16_t counts. No physical conversion is applied in the SPI layer.
• Motor position reset is on-STM32 (via motorPositionReset bitmask + PI_BUFFER_UPDATE_MOTOR_POS_RESET); there are no Pi-side positionOffsets_ in the current code.

DataPublisher::publishSensorData() serialises each field as an LCM message on the raccoon transport. All channel names use the raccoon/ prefix — the old libstp/ prefix is obsolete and will receive no data.

Publish-rate gating

Most IMU channels are capped to 50 Hz with an L-infinity noise-floor epsilon to reduce transport traffic. The accelerometer is intentionally ungated (publishForce) so downstream calibration routines receive every raw frame even when consecutive values are identical.

LCM channel table

All topics below are on raccoon::Transport (raccoon_ring SHM primary, LCM UDP multicast loopback for setup). Message types are the LCM-generated raccoon:: types.

“Retained” channels use publishRetained(). A new subscriber that calls subscribeWithRetain immediately receives the last published value without waiting for the next SPI cycle. This is important for motor position, done flag, and battery voltage, where a late subscriber must not block waiting for the next update.

IMU accuracy channels (gyro/accuracy, accel/accuracy, mag/accuracy, imu/quaternion_accuracy) are published only when the accuracy value changes, not on every loop.

Step 4: raccoon-transport — raccoon_ring SHM

The raccoon-transport library provides the IPC layer between the bridge and all subscribers (raccoon-lib, BotUI, and any diagnostic tool). The primary transport backend is raccoon_ring, a lock-minimal single-producer multi-consumer shared memory ring buffer.

File naming: each LCM channel maps to one file under /dev/shm/. Forward slashes are URL-encoded to _2F, so raccoon/motor/0/power becomes /dev/shm/raccoon_ring_raccoon_2Fmotor_2F0_2Fpower. You can inspect all ring files with ls /dev/shm/raccoon_ring_* and remove them with rm -f /dev/shm/raccoon_ring_* (they are recreated on reader restart).

Ring layout: each file contains a ring_hdr_t (magic 0x52435242, version 3) followed by slot_count slot records of sizeof(slot_t) + max_payload bytes each. Defaults are 64 slots × 2 KiB payload (~130 KiB per channel). The header carries producer_seq (monotonic write counter), wake_seq (futex word), and waiter_count (optimization: producer skips FUTEX_WAKE when 0).

sequenceDiagram
    participant PUB as DataPublisher (bridge)
    participant RING as raccoon_ring file\n(/dev/shm/raccoon_ring_*)
    participant SUB as raccoon-lib spinOnce()

    PUB->>RING: rrb_writer_publish(encoded_lcm, len)\nslot = producer_seq % 64\nslot.seq_lock = 0 (writing flag)\ncopy payload\nslot.seq_lock = producer_seq\nbump producer_seq\nbump wake_seq + FUTEX_WAKE if waiter_count > 0

    SUB->>RING: rrb_reader_recv_wait_many()\nfutex_waitv on wake_seq of all channels
    RING-->>SUB: frame available\ncopy payload out\nverify seq_lock unchanged\ndecode LCM + dispatch callback

SeqLock safety: the producer marks a slot seq = 0 (writing-in-progress), copies the payload, then stamps it with the sequence number. A subscriber that reads seq = 0 after copying knows the producer raced it and skips the frame. Slow subscribers lose frames but never block the producer.

Retained channels: publishRetained() sets a retained=true flag on the publish options. The transport calls rrb_reader_seek_to_latest() when a new subscriber attaches, positioning its read cursor at the most recently published slot so the first rrb_reader_recv() immediately returns the last value — no waiting for the next publish cycle. Battery voltage, motor position, motor done, servo state, odometry, heading, and BEMF are all retained for this reason.

Producer restart transparency: when stm32-data-reader restarts, rrb_writer_create() reinitialises the ring header in place (new producer_seq epoch). Existing subscribers detect the sequence reset and resync. No subscriber restart or file deletion is required.

Namespace isolation: stm32_data_reader.service sets PrivateTmp=false explicitly so the /dev/shm/ files sit in the host-global tmpfs namespace. Any service option that creates a mount namespace (ProtectSystem=strict, PrivateTmp=true) would isolate /dev/shm to the service and make all ring files invisible to raccoon-lib and BotUI.

The lcm-loopback-multicast.service remains required for reliable command delivery (the reliable delivery protocol uses a control channel that currently routes over UDP loopback), but the primary sensor data path uses raccoon_ring SHM exclusively.

Step 5: raccoon-lib HAL

LcmReader runs as a singleton with a background listenLoop thread. This thread calls transport_.spinOnce(0) continuously and updates mutex-protected caches. Each HAL class (Motor, Analog, Digital, IMU) calls the appropriate LcmReader::read*() method which takes the lock and returns the cached value. Sensor reads in user code are non-blocking and always return the most recently observed value.

Command Path (Python → STM32)

Python: motor.set_speed(50)
      │
      ▼
raccoon-lib::hal::motor::Motor::setSpeed()
      │  (maps percent to signed duty, applies inversion)
      ▼
LcmDataWriter::setMotor(port, duty)
      │  (publishes scalar_i32_t to raccoon/motor/N/power_cmd)
      ▼
raccoon_ring SHM → LCM multicast (loopback)
      │
      ▼
stm32-data-reader: CommandSubscriber::onMotorPowerCommand()
      │  (maps percent → duty: duty = percent × 4)
      ▼
DeviceController::setMotorPwm()
      │
      ▼
SpiReal::setMotorPwm()
      │  (calls C API: set_motor_pwm(port, duty))
      ▼
C SPI layer: updates rxBuffer in Pi's memory
      │
      ▼
Next spi_update() call: Pi sends updated rxBuffer to STM32
      │  (full-duplex SPI transaction)
      ▼
STM32: rxBuffer updated by DMA
      │  (SPI completion callback fires)
      ▼
HAL_SPI_TxRxCpltCallback validates version
      │
      ▼
Next BEMF cycle (≤ 1250 µs): update_motor() reads motorControlMode
      │  and motorTarget, calls motor_setDutycycle()
      ▼
TIM1/TIM8 compare register updated
      │
      ▼
Motor PWM changes

The following sequence diagram shows the timing of a motor power command in detail, including where the main loop interleaves with the command subscriber:

sequenceDiagram
    participant PY as Python (raccoon-lib)
    participant RING as raccoon_ring SHM
    participant CS as CommandSubscriber
    participant DC as DeviceController
    participant SPI as SpiReal / C SPI layer
    participant STM as STM32F427

    PY->>RING: publish raccoon/motor/0/power_cmd\n(scalar_i32_t, value=50)
    Note over RING: slot written, wake_seq bumped

    Note over CS: main loop: processMessages() → spinOnce(1ms)
    RING-->>CS: frame delivered, callback fires
    CS->>CS: isTimestampNewer()? yes
    CS->>CS: duty = 50 × 4 = 200
    CS->>DC: setMotorPwm(0, 200)
    DC->>DC: hasSameCommand()? no
    DC->>SPI: setMotorState(0, {Pwm, 200})
    SPI->>SPI: set_motor_pwm(0, 200)\nupdates RxBuffer.motorTarget[0]=200\nRxBuffer.motorControlMode |= MOT_MODE_PWM

    Note over STM: ~5 ms: next spi_update()
    SPI->>STM: ioctl(SPI_IOC_MESSAGE)\nsends RxBuffer with new target
    STM-->>SPI: TxBuffer (updated sensor data)
    Note over STM: HAL_SPI_TxRxCpltCallback\nvalidates TRANSFER_VERSION 21
    Note over STM: next BEMF cycle (≤1250 µs)\nupdate_motor() → motor_setDutycycle(200)\nTIM1/TIM8 CCR updated

LCM command channels

Servo mode vs. servo mode command: The servo mode channel is split into two: raccoon/servo/N/mode (state, reader publishes) and raccoon/servo/N/mode_cmd (command, raccoon-lib publishes). This prevents the reader from subscribing to its own publishes, which previously caused a ~5 ms internal latency floor.

Motor relative move: raccoon/motor/N/relative_cmd carries a delta (BEMF ticks) relative to the motor’s current position. CommandSubscriber::onMotorRelativeCommand reads the current position, adds the delta, and issues an absolute MTP command. The payload is vector3f_t matching the position command format: x=speed_limit, y=delta_ticks.

Reliable Commands

Commands that must not be lost (position targets, PID configuration, servo mode changes, shutdown, position resets) are subscribed with reliableOpts (SubscribeOptions{.reliable = true}). Reliable delivery is a transport-level option implemented in raccoon::Transport; there is no separate ReliablePublisher class. The reliable path adds a sequence number envelope and retransmits until an ACK is received.

Continuous commands (motor power percentage, motor velocity) are sent without reliable delivery to avoid queuing stale commands. If a power command is dropped, the next iteration of the user’s control loop sends a fresh one.

Timestamp-Based Deduplication

The CommandSubscriber tracks the timestamp of the most recently applied command per channel. If a message arrives with a timestamp older than the last applied message (possible if LCM delivers messages out of order), it is silently dropped. This is the isTimestampNewer() check.

Timing Budget

Command latency follows the same path in reverse; the dominant factor is the BEMF cycle (up to 1250 µs per motor in round-robin, up to one full 4-motor rotation = 5 ms in worst case) from when the STM32 receives the new command to when the motor output actually changes.

Related pages

• Firmware Runtime and Scheduling — ISR context, ADC interrupt rates, and super-loop scheduling that produce the data entering the pipeline
• Pi Bridge Internals — deep dive into Application lifecycle, SpiReal vs SpiMock, DataPublisher gate logic, CommandSubscriber, MotorWatchdog, LcmBroker, and raccoon_ring internals
• SPI Communication Protocol — the wire format and all LCM channel names
• Motor Control — what happens inside each BEMF cycle
• IMU Stack — how txBuffer.imu fields are produced by the MPL fusion pipeline
• Robot Services And systemd — how stm32_data_reader.service fits into the full service topology