Edge AI for unmanned vehicles means running a trained machine-learning model on the vehicle, close to its sensors. It can turn camera images or other measurements into detections and decisions without sending every input to a remote server. Its usefulness depends on whether those results arrive early enough, remain accurate in the operating environment, and reach the controller through a well-defined interface.
An onboard object detector supplies one part of an autonomy system. The vehicle still needs to estimate its state, decide what to do, control motion, and handle failures. For engineers and buyers, the decisive question is which part of that chain the AI owns and what the vehicle does when its output becomes unusable.
On This Page
- Where the AI fits in the vehicle
- From sensor measurement to control input
- Measure age of information, not just inference speed
- Computing and model tradeoffs
- Failure modes and recovery
- Applications across air, ground, and water
- What to require from an integrated system
Where the AI fits in the vehicle
Training adjusts a model using examples; inference applies the trained model to new inputs. An onboard deployment can run inference with fixed model parameters throughout a mission. It does not need to learn continuously or contact a cloud service to classify each new image. Raza and colleagues demonstrated this principle with an OpenMV microcontroller integrated into a small aerial vehicle in their TinyML research.
The physical boundary matters. A model running on a nearby ground station or cellular edge server is close to the vehicle geographically but still depends on a communications link. A model running onboard can continue computing after that link drops, provided its local sensors, power, software, and required data remain available. That is a computing capability, not permission to continue the mission regardless of other constraints.
A useful architecture separates three responsibilities:
- Mission computing: interpret observations, maintain tracks, select tasks, and propose motion or payload actions.
- Vehicle control: use state estimates and commanded targets to control the actuators.
- Supervision: decide which commands are permissible and when to transition to another operating mode.
These responsibilities may share hardware, but they need explicit interfaces. PX4's companion-computer documentation describes a separate onboard mission computer connected to the flight controller for computationally demanding functions. That is one concrete implementation pattern, not a universal requirement for every unmanned vehicle.
Cloud systems can support model development and later analysis while the vehicle handles time-sensitive inference. Keeping those roles separate also makes an integration question easier to answer: which functions stop working when the network disappears?
From sensor measurement to control input
Consider an illustrative camera-based ground robot. A perception pipeline acquires an image, applies the resizing and normalization expected by the model, runs inference, and converts the result into objects or labeled image regions. A tracker can associate observations over time; planning software can then use the resulting scene information to propose motion.
Every handoff needs more than a label such as “obstacle.” The receiving component needs to know when the observation was captured, where it is expressed, and what uncertainty accompanies it. Pixel coordinates, a position relative to a camera, and a position in a map are different quantities. Converting an image detection into a physical location requires additional geometry or depth information. A correct classification attached to the wrong time or reference frame can lead to the wrong action.
For integration, define the following agreements before connecting the model to motion:
| Handoff | Information to agree | Why it matters |
|---|---|---|
| Sensor to inference | Capture timestamp, image encoding, calibration, and preprocessing | The model must receive inputs with the intended meaning. |
| Inference to tracking or planning | Class definitions, reference frame, measurement age, confidence, and invalid-result indication | A detection must be interpretable and timely. |
| Planner to controller | Units, frame convention, permitted command type, and command lifetime | The controller must receive a command it supports. |
| Supervisor to mission software | Health conditions, timeout behavior, and authority to resume | Restarting a process must not silently restore control. |
This table is an engineering synthesis of the interface issues illustrated by PX4 offboard control and ROS 2 Jazzy quality-of-service documentation, checked September 8, 2026. It is not a standardized message definition.
ROS 2 quality of service, or QoS, controls message delivery behavior. Its sensor-data profile favors timely samples using best-effort delivery and a smaller queue. Publisher and subscriber policies must also be compatible. Reliable transport alone does not establish that an observation is still useful when the planner consumes it.
PX4 v1.16 provides another useful distinction: “offboard” means outside the flight stack, even when the commanding computer is physically onboard. It accepts supported setpoints through MAVLink or ROS 2, with restrictions on message fields and coordinate frames. Naming a protocol therefore does not establish complete interoperability.
Measure age of information, not just inference speed
A fast model can sit inside a slow system. Camera buffering, preprocessing, memory transfers, inference, postprocessing, scheduling, and command delivery all consume time. NVIDIA's TensorRT benchmarking guidance distinguishes GPU computation from host latency and wider application measurement. An engine benchmark does not include every delay between seeing an obstacle and changing vehicle motion.
For a moving vehicle, measure the time from sensor capture to consumption of the resulting command. Also record how much that time varies, including missed deadlines under competing workloads. Average frames per second can conceal occasional old observations.
An illustrative calculation shows the consequence. Assume a ground robot travels at a constant 2 metres per second while a captured observation takes 0.15 seconds to reach the command stage:
Distance traveled during processing = speed × delay = 2 m/s × 0.15 s = 0.30 m.
The robot travels 30 centimetres during that interval. These are hypothetical inputs, not measured performance. This calculation excludes braking distance, actuator response, perception error, and other margins, so it cannot establish a safe stopping distance.
Queue behavior deserves particular attention. Processing every old frame may be useful for an offline recording task; a motion decision may need the newest usable observation. ROS 2's lifespan policy can expire messages between publication and reception, but the application should still check capture age at the point of use. A transport policy cannot account for every earlier buffer or later processing stage.
Computing and model tradeoffs
Choose hardware against the deployed workload. Peak operations per second describe a processor capability under particular arithmetic assumptions; they do not specify the latency of the complete vehicle application. Measure the actual model, input dimensions, runtime, and concurrent tasks before treating a compute specification as a mission capability.
Power and cooling belong in the same evaluation. NVIDIA's performance best practices identify clocks, power limits, thermal throttling, transfers, and synchronization as factors in reproducible measurements. A short, cool bench run is an insufficient basis for selecting a computer that will operate inside an enclosure for a full mission.
Model changes have their own costs. Lower numerical precision can improve efficiency, but it changes arithmetic. TensorRT's accuracy guidance documents reduced-precision limitations and quantization errors. Recheck the converted model's task results, including consequential misses, rather than assuming a successful conversion preserves behavior.
Use the following selection logic:
- For a narrow recognition task, evaluate whether a compact model on modest compute meets the requirement before adding a larger accelerator.
- For several simultaneous perception tasks, measure contention across the complete pipeline, including memory and sensor handling.
- For delayed inspection results, consider whether post-mission processing meets the need with less onboard computing.
- For motion that depends on perception during a network outage, demonstrate the complete local path and its recovery behavior with the link unavailable.
These are engineering recommendations, not product rankings. The first option is supported in principle by the TinyML aerial demonstration; none establishes that a particular board fits an untested vehicle. The featured photograph shows a historical development board to illustrate physical compute and cooling, not a recommended installation.
Failure modes and recovery
Two different failures need different responses: the model can produce an incorrect interpretation, or the computing and delivery system can fail to provide a usable result.
Confidence does not settle the first problem. Guo and colleagues' calibration research shows that neural-network confidence can be poorly aligned with actual correctness. A high score should not automatically grant control authority. Evaluate errors on representative inputs and conditions; checking only low-confidence outputs can miss confident mistakes.
The following diagnostic table combines that research with the PX4, ROS 2, and TensorRT documentation cited above. Symptoms and proposed checks are engineering interpretation, not reported test results.
| Symptom | Possible failure | Check or response to design |
|---|---|---|
| Confident detections become unreliable in a new environment | The model's learned distinctions do not hold for the new inputs | Evaluate representative conditions and define when the function must be limited or disabled. |
| Detections look correct but commands lag | Buffering or workload contention produces old results | Trace capture-to-command age and reject expired inputs. |
| Performance declines during a mission | Power or thermal limits reduce available compute | Measure sustained operation in the intended enclosure and bound workload. |
| The AI process is running but perception has frozen | A heartbeat remains active without fresh useful output | Monitor freshness and task progress separately from process liveness. |
| A restart produces unexpected motion | Old state or commands regain authority | Require fresh inputs and an explicit control handover before resuming. |
PX4 v1.16 requires an offboard liveness stream above 2 Hz; loss of that stream triggers an exit after the configured timeout, with the response governed by failsafe configuration. This is a communication-health mechanism. It does not prove the incoming perception or setpoints are correct, and 2 Hz is not a recommended perception update rate.
Define degraded behavior for the specific vehicle and environment. A ground robot might stop if stopping is acceptable; an aircraft or underwater vehicle needs a response compatible with its remaining navigation and control capabilities. A fallback that depends on the same failed input offers little protection from that failure.
Applications across air, ground, and water
Airborne recognition: The 2021 TinyML study reports onboard detection using a microcontroller integrated with a DJI Tello. It demonstrates that a narrow inference task need not require a large mission computer. It does not establish general obstacle avoidance, all-weather operation, or performance for other aircraft.
Ground-vehicle perception: In the illustrative robot described earlier, a model could classify objects or segment traversable-looking regions while separate planning and control components handle movement. The integration burden is converting those interpretations into current, geometrically meaningful inputs. A visual category alone cannot establish that terrain is physically traversable by the vehicle.
Underwater observation: MBARI's 2019 report on machine learning for video describes in-situ recognition and tracking of organisms, with model outputs feeding a vehicle control system. This is a documented research application of perception guiding behavior. The same report also discusses analysis of recorded imagery, which should be distinguished from live vehicle control.
Across these applications, classify the proposed function by its consequence: recording a useful image, recommending an action to an operator, or influencing motion. Those uses can share a model while requiring different timing, supervision, and recovery arrangements.
What to require from an integrated system
Ask for a demonstration that follows one observation all the way to the resulting action. It should identify the model and runtime versions, the sensor configuration, the command interface, the measured delay, and the behavior when an input becomes stale or unavailable.
Then require sustained operation with the intended concurrent workloads, representative task-error evaluation, and a controlled recovery demonstration. Agree on acceptable delay and error behavior for the particular task before evaluating the results. As an integration practice, retain the previous model and configuration as a recoverable version; evaluate a model update as a change to system behavior.
Select an edge-AI implementation when it meets those requirements within the vehicle's computing, power, and operational limits. The useful deliverable is a dependable function with a known boundary, whether that function is recognizing an object, tracking marine life, or supplying perception to a motion planner.
Sources
- PX4 v1.16: Companion Computers. Developer documentation describing mission computers connected to flight controllers.
- PX4 v1.16: Offboard Mode. Developer documentation for supported control inputs, liveness, and loss handling.
- ROS 2 Jazzy: Quality of Service settings. Official documentation source covering delivery, freshness, and policy compatibility.
- NVIDIA TensorRT: Performance Benchmarking. Developer guidance distinguishing measurement boundaries and performance influences.
- NVIDIA TensorRT: Best Practices. Developer guidance for measurement and optimization under controlled conditions.
- NVIDIA TensorRT: Accuracy Considerations. Developer guidance on reduced precision and quantization errors.
- Guo et al.: On Calibration of Modern Neural Networks. ICML 2017 research on confidence and correctness.
- Raza et al.: Energy-Efficient Inference on the Edge Exploiting TinyML Capabilities for UAVs. A 2021 research report describing a small-aircraft inference demonstration.
- MBARI: Machine learning offers new insights for analyzing video images. Institutional 2019 research account distinguishing recorded-image analysis and in-situ tracking.
- Featured photograph: NVIDIA Jetson Nano Developer Kit, Thomas Amberg, May 2, 2019. Reused unmodified under Creative Commons Attribution-ShareAlike 2.0; attribution and license terms apply to the photograph.
Last checked: September 8, 2026.
